2024-09-21T01:54:56.652 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T01:54:56.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T01:54:56.757 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/acls} email: pdonnell@redhat.com first_in_suite: false job_id: '7913407' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Reduced data availability - Degraded data redundancy sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWmLErnDFG9xnbGzgX62ASxwVoqJKVXcE/ASwtMkgibIWqpVQ9HWbJrQH4HiewsrxSJxktnQKsuWF1/nFV91kY= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpTc0VuyECkNiSQ9jVWZe5j0kX/uYALbAU8BwMMHubNpeLHk/qVCaaqwrcK3oBS+GMzpwntFdYqevsOFCNT4I8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T01:54:56.757 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T01:54:56.758 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T01:54:56.758 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T01:54:56.759 INFO:teuthology.task.internal:Checking packages... 2024-09-21T01:54:56.777 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T01:54:56.777 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T01:54:56.777 INFO:teuthology.packaging:ref: None 2024-09-21T01:54:56.778 INFO:teuthology.packaging:tag: None 2024-09-21T01:54:56.778 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:54:56.778 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:54:56.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:54:56.920 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120-g18432218-1jammy 2024-09-21T01:54:56.921 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T01:54:56.926 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T01:54:56.926 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T01:54:56.938 INFO:teuthology.task.internal:Saving configuration 2024-09-21T01:54:56.954 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T01:54:56.960 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T01:54:56.986 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:49:14.055165', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIINli1hacoXH1N9JEVfCqzWIqRbjiOAGBcl44+vLEk2T'} 2024-09-21T01:54:57.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:49:14.053127', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILXjmMLaTFsXdTIrZv9cDxvRSLPK7Vxre/vyMxg7sxHC'} 2024-09-21T01:54:57.013 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T01:54:57.026 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T01:54:57.026 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T01:54:57.026 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T01:54:57.093 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efd09fc0160>, signals=[15]) 2024-09-21T01:54:57.094 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T01:54:57.098 INFO:teuthology.task.internal:Opening connections... 2024-09-21T01:54:57.098 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T01:54:57.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:54:57.183 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T01:54:57.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:54:57.258 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T01:54:57.263 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-21T01:54:57.268 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-21T01:54:57.269 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="22.04" 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=jammy 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2024-09-21T01:54:57.320 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2024-09-21T01:54:57.321 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T01:54:57.321 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T01:54:57.321 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T01:54:57.321 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T01:54:57.321 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=jammy 2024-09-21T01:54:57.321 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-21T01:54:57.340 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-21T01:54:57.344 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-21T01:54:57.344 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="22.04" 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=jammy 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T01:54:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T01:54:57.392 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T01:54:57.392 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=jammy 2024-09-21T01:54:57.392 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-21T01:54:57.415 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T01:54:57.423 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T01:54:57.427 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T01:54:57.428 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:54:57.430 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:54:57.439 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T01:54:57.445 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T01:54:57.445 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:54:57.482 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:54:57.569 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T01:54:57.627 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T01:54:57.654 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T01:54:57.654 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T01:54:57.654 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:54:57.655 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:54:57.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:54:57.660 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-21T01:54:57.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:54:57.661 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-21T01:54:57.708 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-56-generic 2024-09-21T01:54:57.708 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.15.0-56-generic 2024-09-21T01:54:57.709 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-21T01:54:57.710 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-56-generic 2024-09-21T01:54:57.710 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.15.0-56-generic 2024-09-21T01:54:57.710 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-09-21T01:54:57.851 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-21T01:54:57.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-21T01:54:57.990 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T01:54:57.990 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T01:54:57.990 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T01:54:57.998 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T01:54:58.025 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T01:54:58.064 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T01:54:58.087 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T01:54:58.106 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T01:54:58.222 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T01:54:58.248 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T01:54:58.295 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T01:54:58.296 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T01:54:58.316 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T01:54:58.334 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T01:54:58.336 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T01:54:58.337 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T01:54:58.351 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T01:54:58.354 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T01:54:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T01:54:58.364 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T01:54:58.371 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T01:54:58.372 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T01:54:58.375 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T01:54:58.375 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T01:54:58.376 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T01:54:58.379 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T01:54:58.379 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T01:54:58.380 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T01:54:58.382 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T01:54:58.384 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T01:54:58.387 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T01:54:58.388 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T01:54:58.388 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T01:54:58.389 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T01:54:58.389 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T01:54:58.390 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T01:54:58.390 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T01:54:58.391 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T01:54:58.391 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T01:54:58.391 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T01:54:58.391 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T01:54:58.401 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T01:54:58.404 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T01:54:58.405 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T01:54:58.405 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T01:54:58.405 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T01:54:58.406 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T01:54:58.406 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T01:54:58.408 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T01:54:58.408 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T01:54:58.411 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T01:54:58.411 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T01:54:58.416 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T01:54:58.422 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T01:54:58.422 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T01:54:58.422 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T01:54:58.423 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T01:54:58.423 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T01:54:58.423 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T01:54:58.425 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T01:54:58.425 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T01:54:58.426 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T01:54:58.441 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T01:54:58.441 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T01:54:58.441 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T01:54:58.441 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T01:54:58.442 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T01:54:58.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T01:54:58.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T01:54:58.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T01:54:58.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T01:54:58.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T01:54:58.555 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T01:54:58.979 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T01:54:58.996 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T01:54:59.004 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T01:54:59.005 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T01:54:59.006 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T01:54:59.102 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T01:54:59.113 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T01:54:59.114 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T01:54:59.129 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T01:54:59.159 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T01:54:59.164 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T01:54:59.165 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T01:54:59.165 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T01:54:59.166 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T01:54:59.245 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T01:55:04.671 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 16.3 MB in 2s (7,530 kB/s) 2024-09-21T01:55:04.708 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 16.3 MB in 2s (7,676 kB/s) 2024-09-21T01:55:06.107 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T01:55:06.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T01:55:06.180 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T01:55:06.313 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T01:55:06.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T01:55:06.377 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-21T01:55:06.378 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-21T01:55:06.394 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T01:55:06.547 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T01:55:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T01:55:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T01:55:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T01:55:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T01:55:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: python2.7-minimal 2024-09-21T01:55:06.548 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T01:55:06.549 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-21T01:55:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:55:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T01:55:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T01:55:06.551 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-21T01:55:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T01:55:06.551 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-21T01:55:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: thermald 2024-09-21T01:55:06.591 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-21T01:55:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:55:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T01:55:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T01:55:06.593 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-09-21T01:55:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T01:55:06.606 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-21T01:55:06.607 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-21T01:55:06.644 INFO:teuthology.orchestra.run.smithi096.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T01:55:06.644 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 113 MB of archives. 2024-09-21T01:55:06.644 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T01:55:06.644 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T01:55:06.785 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T01:55:06.785 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T01:55:06.785 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T01:55:06.786 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T01:55:06.786 INFO:teuthology.orchestra.run.smithi122.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T01:55:06.786 INFO:teuthology.orchestra.run.smithi122.stdout: python2.7-minimal 2024-09-21T01:55:06.786 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T01:55:06.787 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-21T01:55:06.787 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:55:06.787 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T01:55:06.788 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T01:55:06.789 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-21T01:55:06.789 INFO:teuthology.orchestra.run.smithi122.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T01:55:06.789 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-21T01:55:06.789 INFO:teuthology.orchestra.run.smithi122.stdout: thermald 2024-09-21T01:55:06.842 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-21T01:55:06.843 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:55:06.843 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T01:55:06.843 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T01:55:06.844 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-21T01:55:06.845 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T01:55:06.894 INFO:teuthology.orchestra.run.smithi122.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T01:55:06.895 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 113 MB of archives. 2024-09-21T01:55:06.895 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T01:55:06.895 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T01:55:07.043 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T01:55:07.133 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T01:55:07.313 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T01:55:07.409 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T01:55:07.922 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T01:55:07.922 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T01:55:07.922 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T01:55:08.059 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T01:55:08.212 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T01:55:08.212 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T01:55:08.212 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T01:55:08.334 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T01:55:08.432 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T01:55:08.451 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 113 MB in 1s (77.7 MB/s) 2024-09-21T01:55:08.654 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T01:55:08.826 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 113 MB in 2s (71.7 MB/s) 2024-09-21T01:55:09.020 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T01:55:10.728 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T01:55:10.732 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:10.868 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:11.071 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T01:55:11.075 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:11.218 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:13.179 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T01:55:13.201 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:13.296 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:13.436 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T01:55:13.457 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:13.545 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:13.758 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T01:55:13.779 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:13.807 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:14.072 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T01:55:14.093 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:14.131 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:19.702 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:55:19.829 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:55:20.128 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:55:20.249 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:55:20.544 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T01:55:20.566 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T01:55:20.610 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:55:21.501 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:55:21.609 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:55:21.967 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:55:22.121 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:55:22.466 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T01:55:22.488 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T01:55:22.532 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:55:26.345 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T01:55:26.382 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:26.420 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:28.813 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T01:55:28.849 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:55:28.887 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:29.158 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:55:29.297 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:55:29.630 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:55:29.727 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:29.836 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T01:55:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:31.292 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:55:31.292 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T01:55:31.292 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:55:31.292 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:31.458 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:31.627 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:55:31.741 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:55:32.099 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:55:32.246 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:32.372 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T01:55:32.526 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:32.842 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T01:55:32.943 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T01:55:33.069 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:33.934 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:55:33.934 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T01:55:33.934 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:55:33.935 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:34.395 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:34.721 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:34.779 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T01:55:34.779 INFO:teuthology.orchestra.run.smithi096.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:35.796 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T01:55:35.930 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T01:55:36.056 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:37.508 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:55:37.598 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T01:55:37.599 INFO:teuthology.orchestra.run.smithi122.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub' 2024-09-21T01:55:49.660 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T01:55:49.685 INFO:teuthology.orchestra.run.smithi096.stdout:Generating grub configuration file ... 2024-09-21T01:55:49.967 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:55:49.984 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:50.358 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:55:50.363 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T01:55:50.427 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T01:55:50.433 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T01:55:50.605 INFO:teuthology.orchestra.run.smithi096.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T01:55:50.605 INFO:teuthology.orchestra.run.smithi096.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T01:55:50.605 INFO:teuthology.orchestra.run.smithi096.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T01:55:50.657 INFO:teuthology.orchestra.run.smithi096.stdout:done 2024-09-21T01:55:51.124 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-09-21T01:55:51.144 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-09-21T01:55:51.144 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.122.122 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-09-21T01:55:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-09-21T01:55:51.146 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T01:55:51.146 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T01:55:51.146 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T01:55:51.146 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T01:55:51.146 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T01:55:51.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-09-21T01:55:51.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-21T01:55:51.452 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T01:55:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T01:55:51.478 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T01:55:51.573 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T01:55:52.087 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T01:55:52.088 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub' 2024-09-21T01:55:52.109 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T01:55:52.134 INFO:teuthology.orchestra.run.smithi122.stdout:Generating grub configuration file ... 2024-09-21T01:55:52.440 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:55:52.462 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:52.799 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:55:52.805 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T01:55:52.873 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T01:55:52.879 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T01:55:52.959 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T01:55:52.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T01:55:53.032 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T01:55:53.067 INFO:teuthology.orchestra.run.smithi122.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T01:55:53.067 INFO:teuthology.orchestra.run.smithi122.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T01:55:53.067 INFO:teuthology.orchestra.run.smithi122.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T01:55:53.107 INFO:teuthology.orchestra.run.smithi122.stdout:done 2024-09-21T01:55:53.227 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-21T01:55:53.228 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-21T01:55:53.401 INFO:teuthology.orchestra.run.smithi096.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T01:55:53.401 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T01:55:53.402 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T01:55:53.402 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T01:55:53.402 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T01:55:53.402 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T01:55:53.402 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T01:55:53.402 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T01:55:53.442 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T01:55:53.444 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-09-21T01:55:53.502 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-09-21T01:55:53.502 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.122.122 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T01:55:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T01:55:53.504 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-09-21T01:55:53.504 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-09-21T01:55:53.504 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-09-21T01:55:53.504 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-09-21T01:55:53.504 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-21T01:55:53.548 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.YfzwgqcoMI 2024-09-21T01:55:53.548 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YfzwgqcoMI 2024-09-21T01:55:53.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.YfzwgqcoMI 2024-09-21T01:55:53.608 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-21T01:55:53.635 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-21T01:55:53.635 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.122.122 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-21T01:55:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-21T01:55:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-21T01:55:53.637 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T01:55:53.637 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T01:55:53.637 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T01:55:53.637 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T01:55:53.637 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T01:55:53.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-21T01:55:53.716 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.YfzwgqcoMI is 10KB 2024-09-21T01:55:53.769 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.YfzwgqcoMI 2024-09-21T01:55:53.775 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T01:55:53.827 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T01:55:53.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T01:55:53.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T01:55:53.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-21T01:55:53.891 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T01:55:53.891 DEBUG:teuthology.orchestra.run.smithi096:> sudo update-grub 2024-09-21T01:55:53.941 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T01:55:53.952 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T01:55:53.970 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T01:55:54.064 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T01:55:55.072 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub' 2024-09-21T01:55:55.073 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T01:55:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-09-21T01:55:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:55:55.352 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:55.402 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T01:55:55.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T01:55:55.486 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T01:55:55.651 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:55:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T01:55:55.701 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T01:55:55.703 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-21T01:55:55.703 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-21T01:55:55.706 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T01:55:55.821 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T01:55:55.821 INFO:teuthology.orchestra.run.smithi096.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T01:55:55.821 INFO:teuthology.orchestra.run.smithi096.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T01:55:55.840 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-09-21T01:55:55.855 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-09-21T01:55:55.896 INFO:teuthology.orchestra.run.smithi122.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T01:55:55.896 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T01:55:55.896 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T01:55:55.896 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T01:55:55.897 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T01:55:55.897 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T01:55:55.897 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T01:55:55.897 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T01:55:55.940 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T01:55:55.942 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T01:55:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.122.122 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-21T01:55:55.955 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-21T01:55:55.955 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-21T01:55:56.003 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.FpSG6zjeaQ 2024-09-21T01:55:56.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub/grub.cfg /tmp/tmp.FpSG6zjeaQ 2024-09-21T01:55:56.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.FpSG6zjeaQ 2024-09-21T01:55:56.166 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.FpSG6zjeaQ is 10KB 2024-09-21T01:55:56.179 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.FpSG6zjeaQ 2024-09-21T01:55:56.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T01:55:56.238 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T01:55:56.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T01:55:56.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T01:55:56.322 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T01:55:56.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo update-grub 2024-09-21T01:55:57.835 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-09-21T01:55:57.837 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T01:55:57.844 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-21T01:55:58.032 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:55:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:55:58.318 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:55:58.324 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T01:55:58.365 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T01:55:58.371 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T01:55:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T01:55:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T01:55:58.511 INFO:teuthology.orchestra.run.smithi122.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T01:55:58.522 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-21T01:55:58.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-21T01:56:25.882 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T01:56:25.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-21T01:56:25.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:56:28.573 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T01:56:28.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T01:56:28.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:56:44.399 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-21T01:56:46.959 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-21T01:56:53.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-21T01:56:53.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:56:55.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T01:56:55.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:56:56.463 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-21T01:56:59.023 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-21T01:57:08.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-21T01:57:08.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:57:11.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T01:57:11.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:57:29.455 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-21T01:57:40.146 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-21T01:57:40.651 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-21T01:57:40.652 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T01:57:40.652 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-21T01:57:40.702 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-122-generic 2024-09-21T01:57:40.702 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T01:57:40.702 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T01:57:40.702 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T01:57:41.704 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T01:57:41.704 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T01:57:41.776 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-09-21T01:57:41.800 DEBUG:teuthology.parallel:result is None 2024-09-21T01:57:44.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T01:57:44.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:57:45.454 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-21T01:57:46.295 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T01:57:46.295 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T01:57:46.295 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-21T01:57:46.342 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-122-generic 2024-09-21T01:57:46.343 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T01:57:46.343 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T01:57:46.343 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T01:57:47.343 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T01:57:47.344 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T01:57:47.895 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-09-21T01:57:47.917 DEBUG:teuthology.parallel:result is None 2024-09-21T01:57:47.917 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T01:57:47.928 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T01:57:47.929 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:57:47.931 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:57:47.936 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T01:57:47.956 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T01:57:48.025 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T01:57:48.025 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:57:48.027 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:57:48.130 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T01:57:48.136 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T01:57:48.136 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:57:48.139 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:57:48.154 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:48.158 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:48.162 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:48.166 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:48.167 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T01:57:48.173 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T01:57:48.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:57:48.207 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:57:48.227 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T01:57:48.240 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T01:57:48.240 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:57:48.263 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:57:48.274 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:57:48.314 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:57:48.362 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T01:57:48.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:57:48.439 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:57:48.444 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:57:48.490 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T01:57:48.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:57:48.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-21T01:57:48.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-21T01:57:48.677 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T01:57:48.692 INFO:teuthology.task.internal:Starting timer... 2024-09-21T01:57:48.693 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T01:57:48.713 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T01:57:48.727 DEBUG:teuthology.task.selinux:Excluding smithi096: OS 'ubuntu' does not support SELinux 2024-09-21T01:57:48.727 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-09-21T01:57:48.727 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T01:57:48.727 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T01:57:48.727 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T01:57:48.727 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T01:57:48.752 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T01:57:48.754 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T01:57:48.880 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T01:57:48.890 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T01:57:48.890 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T02:05:14.614 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-09-21T02:05:14.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-21T02:05:14.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:05:14.695 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-21T02:05:14.770 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-21T02:05:14.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T02:05:14.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:05:14.850 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-21T02:05:14.926 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T02:05:14.927 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T02:05:14.937 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T02:05:14.937 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:05:14.938 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:05:14.940 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:05:14.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:05:14.968 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T02:05:14.968 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Command line: ntpd -gq 2024-09-21T02:05:14.968 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: ---------------------------------------------------- 2024-09-21T02:05:14.968 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T02:05:14.968 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T02:05:14.969 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: corporation. Support and training for ntp-4 are 2024-09-21T02:05:14.969 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: available at https://www.nwtime.org/support 2024-09-21T02:05:14.969 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: ---------------------------------------------------- 2024-09-21T02:05:14.970 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: proto: precision = 0.044 usec (-24) 2024-09-21T02:05:14.970 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: basedate set to 2022-02-04 2024-09-21T02:05:14.970 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: gps base set to 2022-02-06 (week 2196) 2024-09-21T02:05:14.970 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:05:14.970 INFO:teuthology.orchestra.run.smithi096.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:05:14.970 INFO:teuthology.orchestra.run.smithi096.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:05:14.971 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:05:14.971 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T02:05:14.971 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T02:05:14.971 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T02:05:14.972 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listen normally on 3 enp3s0f1 172.21.15.96:123 2024-09-21T02:05:14.972 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listen normally on 4 lo [::1]:123 2024-09-21T02:05:14.972 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc0d%5]:123 2024-09-21T02:05:14.972 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:14 ntpd[18464]: Listening on routing socket on fd #22 for interface updates 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Command line: ntpd -gq 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: ---------------------------------------------------- 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: corporation. Support and training for ntp-4 are 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: available at https://www.nwtime.org/support 2024-09-21T02:05:14.995 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: ---------------------------------------------------- 2024-09-21T02:05:14.996 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: proto: precision = 0.056 usec (-24) 2024-09-21T02:05:14.996 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: basedate set to 2022-02-04 2024-09-21T02:05:14.996 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: gps base set to 2022-02-06 (week 2196) 2024-09-21T02:05:14.996 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:05:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:05:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listen normally on 4 lo [::1]:123 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-09-21T02:05:14.998 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:14 ntpd[18496]: Listening on routing socket on fd #22 for interface updates 2024-09-21T02:05:15.968 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:15 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:15.968 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:15 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:15.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:15 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:15.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:15 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:15.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:15 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:15.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:15 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:16.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:16 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:16.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:16 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:16.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:16 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:16.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:16 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:16.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:16 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:16.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:16 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:17 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:17 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:17 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:17 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:17 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:17 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:17 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:17 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:18 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:18 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:18 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:18 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:18 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:18.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:18 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:18 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:19.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:19.971 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:19.971 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:19.971 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:19.971 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:19 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:20.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:20 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:20 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:20 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:20 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:20 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:20.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:20 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:20 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:21.970 INFO:teuthology.orchestra.run.smithi096.stdout:21 Sep 02:05:21 ntpd[18464]: ntpd: time slew -0.001216 s 2024-09-21T02:05:21.970 INFO:teuthology.orchestra.run.smithi096.stdout:ntpd: time slew -0.001216s 2024-09-21T02:05:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T02:05:21.971 INFO:teuthology.orchestra.run.smithi096.stderr:21 Sep 02:05:21 ntpd[18464]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T02:05:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:05:22.019 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-09-21T02:05:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:22.996 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:05:22.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:05:22.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:05:22.997 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:05:22.997 INFO:teuthology.orchestra.run.smithi122.stdout:21 Sep 02:05:22 ntpd[18496]: ntpd: time slew -0.002738 s 2024-09-21T02:05:22.997 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew -0.002738s 2024-09-21T02:05:22.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T02:05:22.998 INFO:teuthology.orchestra.run.smithi122.stderr:21 Sep 02:05:22 ntpd[18496]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T02:05:23.043 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:05:23.043 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-21T02:05:23.043 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:23.044 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:23.044 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:23.044 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:05:23.045 INFO:teuthology.run_tasks:Running task install... 2024-09-21T02:05:23.053 DEBUG:teuthology.task.install:project ceph 2024-09-21T02:05:23.053 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T02:05:23.053 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T02:05:23.053 INFO:teuthology.task.install:Using flavor: default 2024-09-21T02:05:23.058 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T02:05:23.058 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T02:05:23.058 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-key list | grep Ceph 2024-09-21T02:05:23.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-09-21T02:05:23.121 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T02:05:23.144 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T02:05:23.151 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T02:05:23.151 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T02:05:23.152 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-21T02:05:23.152 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-21T02:05:23.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:05:23.171 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T02:05:23.171 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T02:05:23.171 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-21T02:05:23.171 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-21T02:05:23.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:05:23.293 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default/ 2024-09-21T02:05:23.293 INFO:teuthology.task.install.deb:Package version is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:05:23.309 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default/ 2024-09-21T02:05:23.309 INFO:teuthology.task.install.deb:Package version is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:05:23.404 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:05:23.404 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T02:05:23.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-21T02:05:23.420 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:05:23.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T02:05:23.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-21T02:05:23.603 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T02:05:23.641 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T02:05:23.656 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T02:05:23.662 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-21T02:05:23.678 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-21T02:05:23.685 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T02:05:23.700 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T02:05:23.716 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T02:05:23.716 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T02:05:23.717 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-21T02:05:23.721 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,432 B] 2024-09-21T02:05:23.728 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T02:05:23.745 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,432 B] 2024-09-21T02:05:23.749 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-21T02:05:23.788 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T02:05:23.804 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T02:05:23.870 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T02:05:23.885 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T02:05:24.031 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 28.6 kB in 0s (69.6 kB/s) 2024-09-21T02:05:24.091 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 28.6 kB in 0s (58.7 kB/s) 2024-09-21T02:05:25.086 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T02:05:25.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-5120-g18432218-1jammy cephadm=19.3.0-5120-g18432218-1jammy ceph-mds=19.3.0-5120-g18432218-1jammy ceph-mgr=19.3.0-5120-g18432218-1jammy ceph-common=19.3.0-5120-g18432218-1jammy ceph-fuse=19.3.0-5120-g18432218-1jammy ceph-test=19.3.0-5120-g18432218-1jammy ceph-volume=19.3.0-5120-g18432218-1jammy radosgw=19.3.0-5120-g18432218-1jammy python3-rados=19.3.0-5120-g18432218-1jammy python3-rgw=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy python3-rbd=19.3.0-5120-g18432218-1jammy libcephfs2=19.3.0-5120-g18432218-1jammy libcephfs-dev=19.3.0-5120-g18432218-1jammy librados2=19.3.0-5120-g18432218-1jammy librbd1=19.3.0-5120-g18432218-1jammy rbd-fuse=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy cephfs-shell=19.3.0-5120-g18432218-1jammy cephfs-top=19.3.0-5120-g18432218-1jammy cephfs-mirror=19.3.0-5120-g18432218-1jammy 2024-09-21T02:05:25.148 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T02:05:25.164 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-5120-g18432218-1jammy cephadm=19.3.0-5120-g18432218-1jammy ceph-mds=19.3.0-5120-g18432218-1jammy ceph-mgr=19.3.0-5120-g18432218-1jammy ceph-common=19.3.0-5120-g18432218-1jammy ceph-fuse=19.3.0-5120-g18432218-1jammy ceph-test=19.3.0-5120-g18432218-1jammy ceph-volume=19.3.0-5120-g18432218-1jammy radosgw=19.3.0-5120-g18432218-1jammy python3-rados=19.3.0-5120-g18432218-1jammy python3-rgw=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy python3-rbd=19.3.0-5120-g18432218-1jammy libcephfs2=19.3.0-5120-g18432218-1jammy libcephfs-dev=19.3.0-5120-g18432218-1jammy librados2=19.3.0-5120-g18432218-1jammy librbd1=19.3.0-5120-g18432218-1jammy rbd-fuse=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy cephfs-shell=19.3.0-5120-g18432218-1jammy cephfs-top=19.3.0-5120-g18432218-1jammy cephfs-mirror=19.3.0-5120-g18432218-1jammy 2024-09-21T02:05:25.165 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T02:05:25.226 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T02:05:25.387 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-21T02:05:25.387 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-21T02:05:25.418 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-21T02:05:25.418 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-21T02:05:25.586 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:05:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:05:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:05:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:05:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:05:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:05:25.587 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:05:25.588 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-21T02:05:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T02:05:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T02:05:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:05:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T02:05:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-21T02:05:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-21T02:05:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:05:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:05:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-21T02:05:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-21T02:05:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-21T02:05:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-09-21T02:05:25.602 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:05:25.602 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:05:25.602 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:05:25.603 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:05:25.603 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:05:25.603 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:05:25.603 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:05:25.604 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-21T02:05:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T02:05:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T02:05:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:05:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T02:05:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-21T02:05:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-21T02:05:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T02:05:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T02:05:25.608 INFO:teuthology.orchestra.run.smithi096.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T02:05:25.608 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-21T02:05:25.608 INFO:teuthology.orchestra.run.smithi096.stdout: btrfs-tools 2024-09-21T02:05:25.700 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-21T02:05:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T02:05:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T02:05:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T02:05:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-21T02:05:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:05:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T02:05:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T02:05:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: socat unzip xmlstarlet zip 2024-09-21T02:05:25.713 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-21T02:05:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T02:05:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T02:05:25.713 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T02:05:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T02:05:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: socat unzip xmlstarlet zip 2024-09-21T02:05:25.774 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 125 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T02:05:25.774 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 239 MB of archives. 2024-09-21T02:05:25.775 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-21T02:05:25.775 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-21T02:05:25.792 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T02:05:25.802 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-21T02:05:25.843 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T02:05:25.845 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 125 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T02:05:25.845 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 239 MB of archives. 2024-09-21T02:05:25.845 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-21T02:05:25.845 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-5120-g18432218-1jammy [3,591 kB] 2024-09-21T02:05:25.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-5120-g18432218-1jammy [3,591 kB] 2024-09-21T02:05:25.864 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T02:05:25.867 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T02:05:25.877 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T02:05:25.889 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-21T02:05:25.927 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T02:05:25.931 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T02:05:25.943 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T02:05:25.946 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-21T02:05:25.947 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T02:05:25.948 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T02:05:25.956 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T02:05:25.958 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T02:05:25.960 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T02:05:25.961 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T02:05:25.964 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T02:05:25.967 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T02:05:25.972 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T02:05:25.979 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T02:05:25.981 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T02:05:25.981 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T02:05:25.982 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T02:05:25.983 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T02:05:25.983 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T02:05:25.984 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T02:05:25.984 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T02:05:25.998 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T02:05:25.999 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T02:05:26.015 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T02:05:26.018 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T02:05:26.018 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-21T02:05:26.019 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T02:05:26.019 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T02:05:26.019 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T02:05:26.020 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T02:05:26.021 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T02:05:26.021 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T02:05:26.032 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T02:05:26.033 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T02:05:26.038 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-5120-g18432218-1jammy [3,370 kB] 2024-09-21T02:05:26.050 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T02:05:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T02:05:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T02:05:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T02:05:26.056 INFO:teuthology.orchestra.run.smithi096.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T02:05:26.057 INFO:teuthology.orchestra.run.smithi096.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T02:05:26.059 INFO:teuthology.orchestra.run.smithi096.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T02:05:26.060 INFO:teuthology.orchestra.run.smithi096.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T02:05:26.064 INFO:teuthology.orchestra.run.smithi096.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-5120-g18432218-1jammy [3,370 kB] 2024-09-21T02:05:26.066 INFO:teuthology.orchestra.run.smithi096.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T02:05:26.068 INFO:teuthology.orchestra.run.smithi096.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T02:05:26.083 INFO:teuthology.orchestra.run.smithi096.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T02:05:26.084 INFO:teuthology.orchestra.run.smithi096.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T02:05:26.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T02:05:26.091 INFO:teuthology.orchestra.run.smithi096.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T02:05:26.117 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-5120-g18432218-1jammy [733 kB] 2024-09-21T02:05:26.130 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-5120-g18432218-1jammy [356 kB] 2024-09-21T02:05:26.142 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-5120-g18432218-1jammy [32.8 kB] 2024-09-21T02:05:26.142 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-5120-g18432218-1jammy [180 kB] 2024-09-21T02:05:26.156 INFO:teuthology.orchestra.run.smithi096.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T02:05:26.157 INFO:teuthology.orchestra.run.smithi096.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T02:05:26.159 INFO:teuthology.orchestra.run.smithi096.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T02:05:26.159 INFO:teuthology.orchestra.run.smithi096.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T02:05:26.160 INFO:teuthology.orchestra.run.smithi096.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T02:05:26.163 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-5120-g18432218-1jammy [72.9 kB] 2024-09-21T02:05:26.164 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-5120-g18432218-1jammy [340 kB] 2024-09-21T02:05:26.170 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-5120-g18432218-1jammy [8,845 kB] 2024-09-21T02:05:26.189 INFO:teuthology.orchestra.run.smithi096.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T02:05:26.189 INFO:teuthology.orchestra.run.smithi096.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-5120-g18432218-1jammy [733 kB] 2024-09-21T02:05:26.190 INFO:teuthology.orchestra.run.smithi096.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T02:05:26.190 INFO:teuthology.orchestra.run.smithi096.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T02:05:26.205 INFO:teuthology.orchestra.run.smithi096.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T02:05:26.205 INFO:teuthology.orchestra.run.smithi096.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T02:05:26.206 INFO:teuthology.orchestra.run.smithi096.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T02:05:26.206 INFO:teuthology.orchestra.run.smithi096.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T02:05:26.206 INFO:teuthology.orchestra.run.smithi096.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T02:05:26.207 INFO:teuthology.orchestra.run.smithi096.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T02:05:26.209 INFO:teuthology.orchestra.run.smithi096.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T02:05:26.212 INFO:teuthology.orchestra.run.smithi096.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-5120-g18432218-1jammy [356 kB] 2024-09-21T02:05:26.214 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T02:05:26.219 INFO:teuthology.orchestra.run.smithi096.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-5120-g18432218-1jammy [32.8 kB] 2024-09-21T02:05:26.221 INFO:teuthology.orchestra.run.smithi096.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-5120-g18432218-1jammy [180 kB] 2024-09-21T02:05:26.225 INFO:teuthology.orchestra.run.smithi096.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T02:05:26.225 INFO:teuthology.orchestra.run.smithi096.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-5120-g18432218-1jammy [72.9 kB] 2024-09-21T02:05:26.227 INFO:teuthology.orchestra.run.smithi096.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-5120-g18432218-1jammy [340 kB] 2024-09-21T02:05:26.228 INFO:teuthology.orchestra.run.smithi096.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T02:05:26.229 INFO:teuthology.orchestra.run.smithi096.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T02:05:26.231 INFO:teuthology.orchestra.run.smithi096.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T02:05:26.231 INFO:teuthology.orchestra.run.smithi096.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T02:05:26.235 INFO:teuthology.orchestra.run.smithi096.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T02:05:26.235 INFO:teuthology.orchestra.run.smithi096.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-5120-g18432218-1jammy [8,845 kB] 2024-09-21T02:05:26.237 INFO:teuthology.orchestra.run.smithi096.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-21T02:05:26.237 INFO:teuthology.orchestra.run.smithi096.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T02:05:26.238 INFO:teuthology.orchestra.run.smithi096.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T02:05:26.244 INFO:teuthology.orchestra.run.smithi096.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T02:05:26.256 INFO:teuthology.orchestra.run.smithi096.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T02:05:26.256 INFO:teuthology.orchestra.run.smithi096.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T02:05:26.259 INFO:teuthology.orchestra.run.smithi096.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T02:05:26.259 INFO:teuthology.orchestra.run.smithi096.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T02:05:26.260 INFO:teuthology.orchestra.run.smithi096.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T02:05:26.260 INFO:teuthology.orchestra.run.smithi096.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T02:05:26.261 INFO:teuthology.orchestra.run.smithi096.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T02:05:26.262 INFO:teuthology.orchestra.run.smithi096.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T02:05:26.268 INFO:teuthology.orchestra.run.smithi096.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T02:05:26.278 INFO:teuthology.orchestra.run.smithi096.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T02:05:26.278 INFO:teuthology.orchestra.run.smithi096.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T02:05:26.280 INFO:teuthology.orchestra.run.smithi096.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T02:05:26.281 INFO:teuthology.orchestra.run.smithi096.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T02:05:26.306 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T02:05:26.319 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T02:05:26.369 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T02:05:26.419 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-5120-g18432218-1jammy [112 kB] 2024-09-21T02:05:26.421 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-5120-g18432218-1jammy [478 kB] 2024-09-21T02:05:26.428 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-5120-g18432218-1jammy [29.1 MB] 2024-09-21T02:05:26.543 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T02:05:26.554 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T02:05:26.578 INFO:teuthology.orchestra.run.smithi096.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-5120-g18432218-1jammy [112 kB] 2024-09-21T02:05:26.580 INFO:teuthology.orchestra.run.smithi096.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-5120-g18432218-1jammy [478 kB] 2024-09-21T02:05:26.580 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T02:05:26.585 INFO:teuthology.orchestra.run.smithi096.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-5120-g18432218-1jammy [29.1 MB] 2024-09-21T02:05:26.588 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-21T02:05:26.590 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T02:05:26.592 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T02:05:26.609 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T02:05:26.614 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T02:05:26.618 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T02:05:26.625 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T02:05:26.703 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T02:05:26.707 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T02:05:26.712 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T02:05:26.714 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T02:05:26.716 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T02:05:26.716 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T02:05:26.718 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T02:05:26.719 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T02:05:26.719 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T02:05:26.789 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T02:05:26.802 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T02:05:26.803 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T02:05:26.803 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T02:05:26.809 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T02:05:26.810 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T02:05:26.871 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T02:05:26.872 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T02:05:26.873 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T02:05:26.874 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T02:05:26.953 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T02:05:26.955 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T02:05:26.957 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T02:05:26.958 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T02:05:26.968 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T02:05:27.034 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T02:05:27.035 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T02:05:27.036 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T02:05:27.039 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T02:05:27.043 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T02:05:27.116 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T02:05:27.118 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T02:05:27.120 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T02:05:27.125 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T02:05:27.126 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T02:05:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T02:05:27.202 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T02:05:27.370 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T02:05:27.372 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T02:05:27.375 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T02:05:27.375 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T02:05:27.378 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T02:05:27.414 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T02:05:27.491 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T02:05:27.492 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-5120-g18432218-1jammy [5,023 kB] 2024-09-21T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T02:05:27.558 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T02:05:27.559 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T02:05:27.560 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T02:05:27.560 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T02:05:27.619 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T02:05:27.627 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T02:05:27.638 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-5120-g18432218-1jammy [252 kB] 2024-09-21T02:05:27.639 INFO:teuthology.orchestra.run.smithi096.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-5120-g18432218-1jammy [5,023 kB] 2024-09-21T02:05:27.641 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-5120-g18432218-1jammy [126 kB] 2024-09-21T02:05:27.642 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-5120-g18432218-1jammy [1,552 kB] 2024-09-21T02:05:27.656 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-5120-g18432218-1jammy [6,198 kB] 2024-09-21T02:05:27.701 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T02:05:27.704 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T02:05:27.706 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T02:05:27.715 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T02:05:27.717 INFO:teuthology.orchestra.run.smithi096.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-5120-g18432218-1jammy [252 kB] 2024-09-21T02:05:27.720 INFO:teuthology.orchestra.run.smithi096.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-5120-g18432218-1jammy [126 kB] 2024-09-21T02:05:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-21T02:05:27.722 INFO:teuthology.orchestra.run.smithi096.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-5120-g18432218-1jammy [1,552 kB] 2024-09-21T02:05:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T02:05:27.725 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T02:05:27.726 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T02:05:27.771 INFO:teuthology.orchestra.run.smithi096.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-5120-g18432218-1jammy [6,198 kB] 2024-09-21T02:05:27.782 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T02:05:27.784 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T02:05:27.820 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-5120-g18432218-1jammy [22.9 MB] 2024-09-21T02:05:27.864 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T02:05:27.864 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T02:05:27.866 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T02:05:27.866 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T02:05:27.870 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T02:05:27.872 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T02:05:27.889 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T02:05:27.890 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T02:05:27.945 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T02:05:27.950 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T02:05:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-5120-g18432218-1jammy [22.9 MB] 2024-09-21T02:05:28.027 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T02:05:28.639 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-5120-g18432218-1jammy [14.0 kB] 2024-09-21T02:05:28.640 INFO:teuthology.orchestra.run.smithi122.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-5120-g18432218-1jammy [865 kB] 2024-09-21T02:05:28.659 INFO:teuthology.orchestra.run.smithi122.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-5120-g18432218-1jammy [2,390 kB] 2024-09-21T02:05:28.699 INFO:teuthology.orchestra.run.smithi122.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-5120-g18432218-1jammy [813 kB] 2024-09-21T02:05:28.709 INFO:teuthology.orchestra.run.smithi122.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-5120-g18432218-1jammy [160 kB] 2024-09-21T02:05:28.711 INFO:teuthology.orchestra.run.smithi122.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-5120-g18432218-1jammy [61.5 MB] 2024-09-21T02:05:28.793 INFO:teuthology.orchestra.run.smithi096.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-5120-g18432218-1jammy [14.0 kB] 2024-09-21T02:05:28.794 INFO:teuthology.orchestra.run.smithi096.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-5120-g18432218-1jammy [865 kB] 2024-09-21T02:05:28.811 INFO:teuthology.orchestra.run.smithi096.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-5120-g18432218-1jammy [2,390 kB] 2024-09-21T02:05:28.851 INFO:teuthology.orchestra.run.smithi096.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-5120-g18432218-1jammy [813 kB] 2024-09-21T02:05:28.863 INFO:teuthology.orchestra.run.smithi096.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-5120-g18432218-1jammy [160 kB] 2024-09-21T02:05:28.866 INFO:teuthology.orchestra.run.smithi096.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-5120-g18432218-1jammy [61.5 MB] 2024-09-21T02:05:30.922 INFO:teuthology.orchestra.run.smithi122.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-5120-g18432218-1jammy [8,625 kB] 2024-09-21T02:05:31.034 INFO:teuthology.orchestra.run.smithi122.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-5120-g18432218-1jammy [14.2 kB] 2024-09-21T02:05:31.034 INFO:teuthology.orchestra.run.smithi122.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-5120-g18432218-1jammy [40.9 MB] 2024-09-21T02:05:31.279 INFO:teuthology.orchestra.run.smithi096.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-5120-g18432218-1jammy [8,625 kB] 2024-09-21T02:05:31.502 INFO:teuthology.orchestra.run.smithi096.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-5120-g18432218-1jammy [14.2 kB] 2024-09-21T02:05:31.503 INFO:teuthology.orchestra.run.smithi096.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-5120-g18432218-1jammy [40.9 MB] 2024-09-21T02:05:32.194 INFO:teuthology.orchestra.run.smithi122.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-5120-g18432218-1jammy [130 kB] 2024-09-21T02:05:32.197 INFO:teuthology.orchestra.run.smithi122.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-5120-g18432218-1jammy [237 kB] 2024-09-21T02:05:32.201 INFO:teuthology.orchestra.run.smithi122.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-5120-g18432218-1jammy [34.0 kB] 2024-09-21T02:05:32.202 INFO:teuthology.orchestra.run.smithi122.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-5120-g18432218-1jammy [23.6 kB] 2024-09-21T02:05:32.203 INFO:teuthology.orchestra.run.smithi122.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-5120-g18432218-1jammy [39.0 kB] 2024-09-21T02:05:32.204 INFO:teuthology.orchestra.run.smithi122.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-5120-g18432218-1jammy [17.6 MB] 2024-09-21T02:05:32.749 INFO:teuthology.orchestra.run.smithi122.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-5120-g18432218-1jammy [93.1 kB] 2024-09-21T02:05:33.128 INFO:teuthology.orchestra.run.smithi096.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-5120-g18432218-1jammy [130 kB] 2024-09-21T02:05:33.131 INFO:teuthology.orchestra.run.smithi096.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-5120-g18432218-1jammy [237 kB] 2024-09-21T02:05:33.135 INFO:teuthology.orchestra.run.smithi096.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-5120-g18432218-1jammy [34.0 kB] 2024-09-21T02:05:33.137 INFO:teuthology.orchestra.run.smithi096.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-5120-g18432218-1jammy [23.6 kB] 2024-09-21T02:05:33.137 INFO:teuthology.orchestra.run.smithi096.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-5120-g18432218-1jammy [39.0 kB] 2024-09-21T02:05:33.139 INFO:teuthology.orchestra.run.smithi096.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-5120-g18432218-1jammy [17.6 MB] 2024-09-21T02:05:33.375 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 239 MB in 7s (34.0 MB/s) 2024-09-21T02:05:33.475 INFO:teuthology.orchestra.run.smithi096.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-5120-g18432218-1jammy [93.1 kB] 2024-09-21T02:05:33.498 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-21T02:05:33.596 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T02:05:33.601 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-21T02:05:33.647 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-21T02:05:33.900 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T02:05:33.927 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T02:05:33.958 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:05:34.080 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 239 MB in 8s (30.9 MB/s) 2024-09-21T02:05:34.189 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-21T02:05:34.270 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T02:05:34.275 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-21T02:05:34.287 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cmd2. 2024-09-21T02:05:34.313 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-21T02:05:34.314 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-21T02:05:34.353 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-21T02:05:34.557 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T02:05:34.584 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T02:05:34.624 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:05:34.698 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T02:05:34.725 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T02:05:34.807 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:05:34.952 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cmd2. 2024-09-21T02:05:34.978 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-21T02:05:35.018 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-21T02:05:35.186 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T02:05:35.213 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T02:05:35.253 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:05:35.363 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T02:05:35.389 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T02:05:35.472 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:05:35.539 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T02:05:35.566 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T02:05:35.597 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:05:35.825 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T02:05:35.891 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:05:36.026 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T02:05:36.053 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:05:36.101 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:05:36.186 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T02:05:36.201 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T02:05:36.236 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:05:36.505 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T02:05:36.532 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:05:36.572 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:05:36.715 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T02:05:36.730 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:05:36.781 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:05:36.875 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T02:05:36.901 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:05:36.942 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:05:37.192 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T02:05:37.219 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:05:37.259 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:05:37.296 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T02:05:37.322 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T02:05:37.362 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:05:37.546 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T02:05:37.573 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:05:37.604 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:05:37.682 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librados2. 2024-09-21T02:05:37.709 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-librados2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:37.749 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:37.940 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T02:05:37.966 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T02:05:38.007 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:05:38.193 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libnbd0. 2024-09-21T02:05:38.220 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-21T02:05:38.260 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-21T02:05:38.484 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librados2. 2024-09-21T02:05:38.510 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../010-librados2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:38.547 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librbd1. 2024-09-21T02:05:38.562 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-librbd1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:38.605 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:38.617 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:39.058 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T02:05:39.073 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:39.087 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libnbd0. 2024-09-21T02:05:39.108 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:39.113 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-21T02:05:39.154 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-21T02:05:39.369 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-09-21T02:05:39.384 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-python3-rados_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:39.428 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:39.448 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librbd1. 2024-09-21T02:05:39.474 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../012-librbd1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:39.514 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:39.756 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T02:05:39.782 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:05:39.822 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:39.984 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T02:05:40.010 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:40.050 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:40.075 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T02:05:40.090 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:40.133 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:40.345 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rados. 2024-09-21T02:05:40.360 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../014-python3-rados_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:40.403 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:40.403 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T02:05:40.430 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:05:40.470 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:40.714 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T02:05:40.740 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:05:40.772 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:40.782 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T02:05:40.808 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T02:05:40.848 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T02:05:40.999 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T02:05:41.025 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:41.066 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:41.101 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T02:05:41.117 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:41.159 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:41.377 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T02:05:41.403 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:05:41.444 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:41.480 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T02:05:41.506 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T02:05:41.546 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:05:41.772 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T02:05:41.798 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T02:05:41.807 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T02:05:41.823 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T02:05:41.838 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T02:05:41.866 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:05:42.107 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T02:05:42.134 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:42.169 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T02:05:42.174 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:42.184 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T02:05:42.227 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:05:42.494 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T02:05:42.520 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T02:05:42.564 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua5.1. 2024-09-21T02:05:42.568 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:05:42.580 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T02:05:42.623 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:05:42.813 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T02:05:42.829 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T02:05:42.863 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:05:42.918 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-any. 2024-09-21T02:05:42.944 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-21T02:05:42.984 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T02:05:43.174 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T02:05:43.189 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T02:05:43.232 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:05:43.237 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package zip. 2024-09-21T02:05:43.264 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-21T02:05:43.304 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T02:05:43.599 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package unzip. 2024-09-21T02:05:43.603 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua5.1. 2024-09-21T02:05:43.626 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T02:05:43.628 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T02:05:43.669 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:05:43.691 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:05:43.955 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-any. 2024-09-21T02:05:43.971 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-21T02:05:44.005 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T02:05:44.079 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package luarocks. 2024-09-21T02:05:44.105 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T02:05:44.139 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:05:44.216 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package zip. 2024-09-21T02:05:44.231 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-21T02:05:44.274 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T02:05:44.472 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-09-21T02:05:44.488 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-librgw2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:44.539 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:44.596 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package unzip. 2024-09-21T02:05:44.622 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T02:05:44.663 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:05:45.008 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package luarocks. 2024-09-21T02:05:45.034 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T02:05:45.066 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:05:45.258 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T02:05:45.274 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:45.308 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:45.424 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librgw2. 2024-09-21T02:05:45.450 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../028-librgw2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:45.490 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:45.611 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T02:05:45.638 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T02:05:45.678 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:05:45.990 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T02:05:46.017 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:46.056 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:46.226 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T02:05:46.252 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:46.292 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:46.427 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-09-21T02:05:46.453 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-ceph-common_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:46.493 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:46.637 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T02:05:46.663 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T02:05:46.704 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:05:46.982 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T02:05:47.008 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:47.073 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:47.409 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-common. 2024-09-21T02:05:47.436 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../032-ceph-common_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:47.476 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:48.266 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-09-21T02:05:48.293 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-ceph-base_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:48.342 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:48.937 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T02:05:48.964 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T02:05:49.004 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:05:49.307 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T02:05:49.334 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T02:05:49.339 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-base. 2024-09-21T02:05:49.365 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../033-ceph-base_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:05:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:05:49.415 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:05:49.702 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T02:05:49.728 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T02:05:49.769 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:05:49.977 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T02:05:50.003 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T02:05:50.043 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:05:50.072 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T02:05:50.098 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T02:05:50.138 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:05:50.338 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T02:05:50.364 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T02:05:50.400 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T02:05:50.404 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:05:50.423 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T02:05:50.459 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:05:50.749 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T02:05:50.753 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T02:05:50.776 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T02:05:50.779 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T02:05:50.815 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:05:50.820 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T02:05:51.093 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T02:05:51.119 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T02:05:51.123 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-portend. 2024-09-21T02:05:51.150 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-21T02:05:51.159 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:05:51.190 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T02:05:51.421 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T02:05:51.448 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T02:05:51.451 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T02:05:51.466 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T02:05:51.479 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:05:51.501 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T02:05:51.773 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T02:05:51.796 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T02:05:51.800 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T02:05:51.823 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T02:05:51.840 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T02:05:51.863 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:05:52.143 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-portend. 2024-09-21T02:05:52.169 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-21T02:05:52.192 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T02:05:52.209 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T02:05:52.218 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T02:05:52.250 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T02:05:52.504 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T02:05:52.530 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T02:05:52.554 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T02:05:52.570 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T02:05:52.580 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T02:05:52.620 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T02:05:52.832 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T02:05:52.858 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T02:05:52.898 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:05:52.923 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T02:05:52.950 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T02:05:52.990 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:05:53.227 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T02:05:53.253 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T02:05:53.293 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T02:05:53.293 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-09-21T02:05:53.320 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T02:05:53.360 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:05:53.563 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T02:05:53.589 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T02:05:53.629 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T02:05:53.655 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T02:05:53.681 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T02:05:53.713 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:05:53.932 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T02:05:53.959 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T02:05:53.999 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:05:54.025 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T02:05:54.052 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T02:05:54.092 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:05:54.395 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-09-21T02:05:54.421 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T02:05:54.426 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-mako. 2024-09-21T02:05:54.453 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T02:05:54.461 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:05:54.485 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:05:54.796 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T02:05:54.806 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T02:05:54.823 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T02:05:54.833 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T02:05:54.863 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:05:54.884 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:05:55.168 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T02:05:55.174 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T02:05:55.195 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T02:05:55.200 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T02:05:55.227 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:05:55.266 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:05:55.513 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-09-21T02:05:55.539 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T02:05:55.580 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:05:55.585 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webob. 2024-09-21T02:05:55.611 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T02:05:55.652 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:05:55.917 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T02:05:55.932 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T02:05:55.983 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:05:56.005 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T02:05:56.031 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T02:05:56.075 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:05:56.287 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T02:05:56.313 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T02:05:56.353 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:05:56.374 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T02:05:56.400 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T02:05:56.440 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:05:56.657 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T02:05:56.683 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T02:05:56.743 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-paste. 2024-09-21T02:05:56.748 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T02:05:56.770 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T02:05:56.810 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:05:57.060 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T02:05:57.086 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T02:05:57.126 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:05:57.130 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T02:05:57.156 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T02:05:57.196 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:05:57.482 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T02:05:57.499 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T02:05:57.508 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T02:05:57.525 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T02:05:57.557 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T02:05:57.565 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:05:57.860 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-idna. 2024-09-21T02:05:57.887 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T02:05:57.927 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:05:58.230 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T02:05:58.256 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T02:05:58.296 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:05:58.634 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests. 2024-09-21T02:05:58.660 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T02:05:58.700 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:05:58.753 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T02:05:58.779 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T02:05:58.819 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T02:05:59.012 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T02:05:59.039 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T02:05:59.113 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T02:05:59.140 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T02:05:59.180 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:05:59.470 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:05:59.560 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T02:05:59.586 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T02:05:59.626 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T02:05:59.947 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-idna. 2024-09-21T02:05:59.972 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T02:06:00.005 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:06:00.116 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T02:06:00.142 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:00.191 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:00.291 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T02:06:00.316 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T02:06:00.357 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:06:00.538 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T02:06:00.565 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:00.596 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:00.652 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests. 2024-09-21T02:06:00.679 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T02:06:00.718 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:06:00.865 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T02:06:00.892 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T02:06:00.932 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:06:01.029 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T02:06:01.056 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T02:06:01.096 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:06:01.235 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T02:06:01.262 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T02:06:01.293 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T02:06:01.399 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T02:06:01.425 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:01.465 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T02:06:01.648 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:01.688 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:01.837 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T02:06:01.863 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:01.903 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:02.092 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T02:06:02.119 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:02.159 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:02.173 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T02:06:02.199 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T02:06:02.239 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:06:02.559 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T02:06:02.585 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T02:06:02.625 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T02:06:02.738 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T02:06:02.764 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:02.804 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T02:06:02.988 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:03.036 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:03.423 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T02:06:03.449 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:03.481 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:03.924 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-09-21T02:06:03.951 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:03.991 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:04.118 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T02:06:04.144 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:04.184 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:04.227 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T02:06:04.242 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:04.285 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:04.647 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T02:06:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:04.714 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-09-21T02:06:05.210 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-cephadm_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:05.250 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:05.504 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph. 2024-09-21T02:06:05.531 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../069-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:05.545 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T02:06:05.571 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:05.571 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T02:06:05.603 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T02:06:05.890 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T02:06:05.916 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:05.922 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T02:06:05.938 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:06:05.956 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:05.981 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:06.268 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T02:06:06.294 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:06.334 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:06.720 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephadm. 2024-09-21T02:06:06.747 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../072-cephadm_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:06.787 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:06.945 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-babel. 2024-09-21T02:06:06.972 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:06:07.011 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:07.106 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T02:06:07.133 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T02:06:07.172 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T02:06:07.331 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T02:06:07.358 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T02:06:07.398 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:06:07.492 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T02:06:07.507 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:06:07.550 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:07.760 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T02:06:07.787 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:07.827 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:08.155 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T02:06:08.182 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T02:06:08.222 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T02:06:08.533 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-09-21T02:06:08.561 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T02:06:08.572 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-babel. 2024-09-21T02:06:08.600 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:06:08.600 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:06:08.638 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:08.920 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T02:06:08.941 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T02:06:08.947 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:08.968 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T02:06:08.986 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:09.007 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:06:09.352 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T02:06:09.380 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:09.419 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:09.722 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T02:06:09.749 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T02:06:09.789 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T02:06:10.091 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-routes. 2024-09-21T02:06:10.118 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T02:06:10.158 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:06:10.503 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T02:06:10.530 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:10.569 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:11.802 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T02:06:11.817 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T02:06:11.860 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:06:12.356 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T02:06:12.383 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T02:06:12.499 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:06:12.838 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T02:06:12.866 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T02:06:12.896 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:06:13.166 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T02:06:13.193 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T02:06:13.232 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:06:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T02:06:13.374 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T02:06:13.415 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:06:13.902 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T02:06:13.929 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T02:06:14.018 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T02:06:14.046 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:14.054 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:06:14.084 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:14.417 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T02:06:14.444 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T02:06:14.499 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:06:14.802 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T02:06:14.830 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T02:06:14.877 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:06:15.046 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T02:06:15.074 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T02:06:15.112 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T02:06:15.374 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T02:06:15.390 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T02:06:15.440 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:06:15.753 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T02:06:15.781 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T02:06:15.819 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:06:15.902 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T02:06:15.931 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:15.969 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:16.081 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T02:06:16.109 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-21T02:06:16.139 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T02:06:16.485 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T02:06:16.513 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T02:06:16.552 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T02:06:16.872 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T02:06:16.900 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T02:06:16.939 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:06:16.964 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T02:06:16.991 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T02:06:17.030 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T02:06:17.250 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T02:06:17.278 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T02:06:17.317 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T02:06:17.333 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T02:06:17.361 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T02:06:17.399 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:06:17.621 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T02:06:17.648 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T02:06:17.687 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:06:17.745 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T02:06:17.772 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T02:06:17.811 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:06:18.122 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T02:06:18.150 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-21T02:06:18.189 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T02:06:18.501 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T02:06:18.529 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T02:06:18.567 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T02:06:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T02:06:18.727 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:18.769 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:18.863 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T02:06:18.890 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T02:06:18.920 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:06:19.055 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T02:06:19.083 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T02:06:19.113 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:06:19.190 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T02:06:19.218 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T02:06:19.256 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T02:06:19.425 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T02:06:19.453 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:06:19.529 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:06:19.701 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T02:06:19.729 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T02:06:19.768 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:06:19.878 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-09-21T02:06:19.906 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:06:19.944 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T02:06:20.189 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-09-21T02:06:20.217 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T02:06:20.247 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:06:20.382 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T02:06:20.399 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:20.440 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:20.493 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T02:06:20.509 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T02:06:20.551 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:06:20.742 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T02:06:20.770 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T02:06:20.826 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:06:20.914 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-09-21T02:06:20.942 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-ceph-test_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:20.980 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:21.121 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T02:06:21.149 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:06:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:06:21.439 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package jq. 2024-09-21T02:06:21.456 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:06:21.498 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T02:06:21.750 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package socat. 2024-09-21T02:06:21.778 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T02:06:21.817 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:06:22.095 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T02:06:22.111 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T02:06:22.145 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:06:22.466 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-test. 2024-09-21T02:06:22.482 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../100-ceph-test_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:22.524 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:23.029 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T02:06:23.045 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:23.087 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:23.400 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-21T02:06:23.428 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:23.458 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:23.778 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-colorama. 2024-09-21T02:06:23.806 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-21T02:06:23.844 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-21T02:06:24.113 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-shell. 2024-09-21T02:06:24.130 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:24.171 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:24.432 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-top. 2024-09-21T02:06:24.449 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:24.490 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:24.538 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T02:06:24.554 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:24.588 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:24.743 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T02:06:24.771 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:24.809 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:24.909 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-21T02:06:24.925 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:24.959 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:25.088 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T02:06:25.116 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T02:06:25.145 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:06:25.262 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-colorama. 2024-09-21T02:06:25.290 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-21T02:06:25.328 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-21T02:06:25.450 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T02:06:25.479 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T02:06:25.516 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:06:25.590 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-shell. 2024-09-21T02:06:25.618 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:25.656 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:25.887 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T02:06:25.916 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T02:06:25.926 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-top. 2024-09-21T02:06:25.953 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:06:25.954 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:06:25.992 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:26.228 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T02:06:26.244 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:26.278 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:26.367 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pkg-config. 2024-09-21T02:06:26.396 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T02:06:26.434 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:06:26.598 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T02:06:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T02:06:26.664 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:06:26.695 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T02:06:26.723 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T02:06:26.761 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:06:26.994 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T02:06:27.022 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T02:06:27.060 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:06:27.165 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T02:06:27.193 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T02:06:27.239 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T02:06:27.373 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T02:06:27.401 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T02:06:27.439 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:06:27.526 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T02:06:27.554 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T02:06:27.592 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T02:06:27.828 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package pkg-config. 2024-09-21T02:06:27.856 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T02:06:27.886 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:06:27.912 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T02:06:27.940 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T02:06:27.979 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:06:28.131 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T02:06:28.147 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T02:06:28.189 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:06:28.249 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T02:06:28.276 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T02:06:28.306 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T02:06:28.592 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T02:06:28.620 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T02:06:28.634 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-09-21T02:06:28.659 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T02:06:28.662 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-21T02:06:28.700 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T02:06:28.962 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T02:06:28.989 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T02:06:28.996 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T02:06:29.024 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T02:06:29.028 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T02:06:29.054 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:06:29.349 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T02:06:29.376 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T02:06:29.415 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:06:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T02:06:29.527 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T02:06:29.566 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:06:29.701 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T02:06:29.717 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T02:06:29.759 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T02:06:29.852 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-toml. 2024-09-21T02:06:29.881 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-21T02:06:29.910 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T02:06:30.121 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-py. 2024-09-21T02:06:30.149 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-21T02:06:30.187 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T02:06:30.196 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T02:06:30.224 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T02:06:30.263 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:06:30.533 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T02:06:30.561 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T02:06:30.591 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T02:06:30.599 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:06:30.619 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T02:06:30.658 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:06:30.939 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T02:06:30.952 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T02:06:30.995 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:06:31.045 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T02:06:31.077 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T02:06:31.112 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:06:31.407 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-toml. 2024-09-21T02:06:31.435 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-21T02:06:31.473 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T02:06:31.748 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-09-21T02:06:31.759 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T02:06:31.776 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../123-radosgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:31.787 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T02:06:31.814 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:31.825 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:06:32.171 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T02:06:32.199 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T02:06:32.237 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:06:32.507 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T02:06:32.535 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T02:06:32.565 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:06:32.934 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T02:06:32.950 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:32.991 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:33.243 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T02:06:33.253 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package radosgw. 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../123-radosgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:33.310 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:33.506 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:06:33.896 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T02:06:34.254 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T02:06:34.482 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T02:06:34.509 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:06:34.547 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:34.682 INFO:teuthology.orchestra.run.smithi122.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T02:06:34.758 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:34.852 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T02:06:35.102 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:06:35.253 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:06:35.258 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-09-21T02:06:35.414 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:06:35.515 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T02:06:35.682 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:06:35.817 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T02:06:35.942 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:06:36.068 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:06:36.252 INFO:teuthology.orchestra.run.smithi096.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T02:06:36.319 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T02:06:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:36.561 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:06:36.679 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T02:06:36.980 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user cephadm....done 2024-09-21T02:06:36.983 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:06:37.120 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:06:37.342 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T02:06:37.438 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:06:37.593 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-21T02:06:37.698 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:06:37.824 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:06:37.835 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T02:06:38.075 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T02:06:38.095 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:06:38.326 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:06:38.380 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:06:38.494 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T02:06:38.639 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:06:38.804 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:06:38.916 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:06:39.165 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T02:06:39.325 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:39.424 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:06:39.684 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T02:06:39.694 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-21T02:06:39.803 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:06:39.921 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-21T02:06:39.968 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:06:40.180 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:06:40.228 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:06:40.331 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:06:40.488 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:06:40.590 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:06:40.748 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:06:40.970 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T02:06:41.005 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T02:06:41.069 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:06:41.157 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:41.195 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T02:06:41.417 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:06:41.551 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-21T02:06:41.580 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:41.702 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:06:41.705 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T02:06:41.836 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-21T02:06:41.973 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:06:42.104 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:06:42.231 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:06:42.241 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:06:42.490 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T02:06:42.677 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:06:42.869 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T02:06:42.904 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T02:06:42.985 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:06:43.136 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T02:06:43.362 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:06:43.504 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:43.613 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:06:43.630 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T02:06:43.739 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:06:43.865 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:06:43.898 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:06:44.157 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:06:44.174 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T02:06:44.450 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T02:06:44.576 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T02:06:44.610 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:06:44.843 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:06:45.203 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:06:45.270 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:06:45.496 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-21T02:06:45.530 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:06:45.656 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:06:45.782 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:06:45.798 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:06:46.091 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T02:06:46.149 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:06:46.400 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:06:46.477 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T02:06:46.526 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:46.736 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:06:46.811 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:06:47.120 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:06:47.120 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:06:47.414 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-21T02:06:47.723 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:06:48.074 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:06:48.317 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:06:48.367 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:06:48.468 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:48.594 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:06:48.728 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T02:06:48.778 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:06:49.096 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:06:49.286 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:06:49.497 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:06:49.623 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:06:49.924 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:06:50.176 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T02:06:50.210 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:06:50.426 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:06:50.462 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:06:50.552 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:06:50.596 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T02:06:50.846 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:06:50.856 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:06:50.965 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:06:51.075 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:06:51.097 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T02:06:51.351 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:06:51.356 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T02:06:51.611 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T02:06:51.615 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:06:51.854 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:06:51.867 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T02:06:51.988 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:06:52.118 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:06:52.227 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T02:06:52.290 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:06:52.503 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:06:52.540 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T02:06:52.638 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:06:52.792 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T02:06:52.922 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:06:53.068 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:06:53.215 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:06:53.319 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T02:06:53.499 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:06:53.595 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:06:53.625 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:06:53.721 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T02:06:53.743 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:53.824 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T02:06:54.006 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:06:54.061 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:06:54.132 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:06:54.187 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:06:54.425 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:06:54.589 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:06:54.734 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:06:54.848 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T02:06:54.974 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T02:06:55.002 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:06:55.162 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:06:55.234 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:06:55.288 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:06:55.343 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:06:55.385 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T02:06:55.615 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:06:55.644 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:06:55.741 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:06:55.920 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:06:56.143 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:06:56.291 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T02:06:56.416 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T02:06:56.427 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T02:06:56.562 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T02:06:56.684 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T02:06:56.821 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:06:56.944 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:06:56.947 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:06:57.203 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:06:57.256 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:06:57.558 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:06:57.588 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T02:06:57.898 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:06:57.945 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T02:06:58.071 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T02:06:58.234 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:06:58.339 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T02:06:58.351 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:58.461 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:58.587 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:06:58.598 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:06:58.841 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:06:58.923 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:59.049 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:59.175 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T02:06:59.243 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T02:06:59.443 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:59.520 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:06:59.578 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:59.712 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:06:59.864 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:06:59.923 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:06:59.923 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:06:59.965 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.066 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.192 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:07:00.397 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.523 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.529 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.649 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.655 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.775 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:00.781 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T02:07:00.901 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:07:01.049 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:01.175 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:01.318 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:01.548 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:07:01.549 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:07:02.087 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.104 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.221 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.230 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.356 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.372 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.482 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.498 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.608 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:02.691 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:07:02.734 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:03.244 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-09-21T02:07:03.604 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-09-21T02:07:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-09-21T02:07:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T02:07:03.920 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T02:07:03.982 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.083 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.209 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T02:07:04.335 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.461 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.596 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.710 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:04.904 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:07:04.904 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout:Adding group ceph....done 2024-09-21T02:07:05.412 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:05.490 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user ceph....done 2024-09-21T02:07:05.538 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:05.650 INFO:teuthology.orchestra.run.smithi096.stdout:Setting system user ceph properties....done 2024-09-21T02:07:05.662 INFO:teuthology.orchestra.run.smithi096.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T02:07:05.797 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T02:07:05.995 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:07:05.995 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:07:06.100 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T02:07:06.431 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:06.555 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:06.785 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T02:07:06.814 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:07:06.814 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:07:07.258 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:07.292 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:07.419 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:07.499 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:07:07.499 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:07:07.878 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:07:07.878 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:07:07.951 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:08.139 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:07:08.139 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:07:08.578 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:08.606 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:08.858 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:07:08.858 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:07:08.962 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T02:07:09.338 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:09.464 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:09.473 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:09.606 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:09.705 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:07:09.706 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:07:09.836 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:07:09.836 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:07:10.133 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:07:10.342 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:10.493 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:10.611 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:10.778 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:10.801 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:11.089 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:07:11.089 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:07:11.131 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:07:11.249 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:07:11.385 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:07:11.611 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:11.754 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:11.905 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:12.118 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:07:12.118 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:07:12.510 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:07:12.583 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:12.733 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:12.851 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:12.986 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:07:13.153 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:07:13.155 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-21T02:07:13.211 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-21T02:07:13.364 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:07:13.405 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-21T02:07:13.406 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-21T02:07:13.490 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:07:13.588 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:07:13.589 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:07:13.589 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:07:13.589 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-21T02:07:13.589 INFO:teuthology.orchestra.run.smithi122.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-21T02:07:13.589 INFO:teuthology.orchestra.run.smithi122.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-21T02:07:13.590 INFO:teuthology.orchestra.run.smithi122.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-21T02:07:13.590 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-21T02:07:13.590 INFO:teuthology.orchestra.run.smithi122.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:07:13.591 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-21T02:07:13.591 INFO:teuthology.orchestra.run.smithi122.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-21T02:07:13.591 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-21T02:07:13.591 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-21T02:07:13.634 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-21T02:07:13.634 INFO:teuthology.orchestra.run.smithi122.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-21T02:07:13.634 INFO:teuthology.orchestra.run.smithi122.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-21T02:07:13.635 INFO:teuthology.orchestra.run.smithi122.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-21T02:07:13.635 INFO:teuthology.orchestra.run.smithi122.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-21T02:07:13.635 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-21T02:07:13.635 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-21T02:07:13.636 INFO:teuthology.orchestra.run.smithi122.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:07:13.637 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-21T02:07:13.637 INFO:teuthology.orchestra.run.smithi122.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-21T02:07:13.642 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:07:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:4 upgraded, 39 newly installed, 0 to remove and 328 not upgraded. 2024-09-21T02:07:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 55.8 MB of archives. 2024-09-21T02:07:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-21T02:07:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-21T02:07:14.393 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:07:14.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-21T02:07:14.474 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-21T02:07:14.598 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-21T02:07:14.599 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-21T02:07:14.600 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-21T02:07:14.628 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-21T02:07:14.630 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-21T02:07:14.633 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-21T02:07:14.636 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-21T02:07:14.637 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-21T02:07:14.639 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-21T02:07:14.702 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-21T02:07:14.702 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-21T02:07:14.703 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-21T02:07:14.783 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-21T02:07:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:07:15.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-21T02:07:15.088 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T02:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-21T02:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-21T02:07:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-21T02:07:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T02:07:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T02:07:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:07:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:07:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-21T02:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-21T02:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-21T02:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-21T02:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-21T02:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-21T02:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-21T02:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-21T02:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-21T02:07:15.562 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-21T02:07:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-21T02:07:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-21T02:07:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-21T02:07:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-21T02:07:15.563 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-21T02:07:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-21T02:07:15.564 INFO:teuthology.orchestra.run.smithi096.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:07:15.565 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-09-21T02:07:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-21T02:07:15.570 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-21T02:07:15.575 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-21T02:07:15.576 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.4 [262 kB] 2024-09-21T02:07:15.582 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-21T02:07:15.583 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-21T02:07:15.593 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-21T02:07:15.597 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-21T02:07:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-21T02:07:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-21T02:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout:4 upgraded, 39 newly installed, 0 to remove and 328 not upgraded. 2024-09-21T02:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 55.8 MB of archives. 2024-09-21T02:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-21T02:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-21T02:07:15.676 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-21T02:07:15.700 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.4 [1,094 kB] 2024-09-21T02:07:15.722 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-21T02:07:15.754 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-21T02:07:15.762 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-21T02:07:15.789 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-21T02:07:15.795 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-21T02:07:15.796 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-21T02:07:15.797 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-21T02:07:15.798 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-21T02:07:15.803 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-21T02:07:15.803 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-21T02:07:15.804 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-21T02:07:15.815 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-21T02:07:15.815 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-21T02:07:15.817 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-21T02:07:15.819 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-21T02:07:15.819 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-21T02:07:15.820 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-21T02:07:15.821 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-21T02:07:15.826 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-21T02:07:15.829 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-21T02:07:15.838 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-21T02:07:15.838 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-21T02:07:15.840 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-21T02:07:16.151 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-21T02:07:16.153 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-21T02:07:16.153 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.4 [262 kB] 2024-09-21T02:07:16.156 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-21T02:07:16.156 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-21T02:07:16.160 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-21T02:07:16.162 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-21T02:07:16.169 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-21T02:07:16.169 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-21T02:07:16.191 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-21T02:07:16.191 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.4 [1,094 kB] 2024-09-21T02:07:16.201 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-21T02:07:16.220 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-21T02:07:16.223 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-21T02:07:16.224 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-21T02:07:16.224 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-21T02:07:16.225 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-21T02:07:16.235 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-21T02:07:16.237 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-21T02:07:16.377 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-21T02:07:16.377 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-21T02:07:16.377 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-21T02:07:16.377 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-21T02:07:16.378 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-21T02:07:16.378 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-21T02:07:16.404 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-21T02:07:16.405 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-21T02:07:16.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-21T02:07:16.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-21T02:07:16.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-21T02:07:16.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-21T02:07:16.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-21T02:07:16.438 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-21T02:07:16.449 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-21T02:07:16.449 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-21T02:07:16.818 INFO:teuthology.orchestra.run.smithi122.stdout:Preconfiguring packages ... 2024-09-21T02:07:16.863 INFO:teuthology.orchestra.run.smithi096.stdout:Preconfiguring packages ... 2024-09-21T02:07:16.977 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 55.8 MB in 3s (20.2 MB/s) 2024-09-21T02:07:17.020 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 55.8 MB in 1s (63.7 MB/s) 2024-09-21T02:07:17.300 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:07:17.306 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:07:17.339 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:07:17.345 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:07:17.434 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:07:17.559 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:07:18.032 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-21T02:07:18.189 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-21T02:07:18.944 INFO:teuthology.orchestra.run.smithi122.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-21T02:07:19.179 INFO:teuthology.orchestra.run.smithi096.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-21T02:07:19.215 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package flex. 2024-09-21T02:07:19.309 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-21T02:07:19.355 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-21T02:07:19.423 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package flex. 2024-09-21T02:07:19.520 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:07:19.525 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-21T02:07:19.571 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-21T02:07:19.889 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:07:20.070 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:07:20.286 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-21T02:07:20.681 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:07:21.190 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:07:21.216 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:07:21.222 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:07:21.343 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:07:21.682 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:07:21.687 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:07:21.770 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:07:21.803 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:07:21.917 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:07:22.222 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:07:22.303 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:07:22.352 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:07:22.672 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:07:22.779 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package bison. 2024-09-21T02:07:22.883 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:07:22.889 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-21T02:07:22.939 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:07:23.053 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package bison. 2024-09-21T02:07:23.155 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:07:23.161 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-21T02:07:23.206 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:07:23.323 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-21T02:07:23.352 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-21T02:07:23.390 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:07:23.606 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-21T02:07:23.634 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-21T02:07:23.673 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:07:23.693 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package dump. 2024-09-21T02:07:23.721 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-21T02:07:23.760 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking dump (0.4b47-3) ... 2024-09-21T02:07:24.010 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package dump. 2024-09-21T02:07:24.037 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-21T02:07:24.076 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking dump (0.4b47-3) ... 2024-09-21T02:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package indent. 2024-09-21T02:07:24.097 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:24.139 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:07:24.338 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package indent. 2024-09-21T02:07:24.354 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:24.388 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:07:24.450 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-21T02:07:24.466 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-21T02:07:24.508 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:07:24.649 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-21T02:07:24.677 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-21T02:07:24.716 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-21T02:07:24.882 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-21T02:07:24.921 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:07:25.045 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-21T02:07:25.073 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-21T02:07:25.111 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:07:25.173 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-21T02:07:25.201 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-21T02:07:25.240 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:07:25.364 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-21T02:07:25.392 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-21T02:07:25.431 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:07:25.610 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-21T02:07:25.638 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-21T02:07:25.677 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:25.768 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-21T02:07:25.796 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-21T02:07:25.834 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:25.930 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-21T02:07:25.946 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-21T02:07:25.988 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:26.179 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-21T02:07:26.207 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-21T02:07:26.241 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjson-perl. 2024-09-21T02:07:26.245 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:26.269 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-21T02:07:26.308 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-21T02:07:26.473 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjson-perl. 2024-09-21T02:07:26.489 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-21T02:07:26.531 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-21T02:07:26.835 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-21T02:07:26.853 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-21T02:07:26.863 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-21T02:07:26.881 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-21T02:07:26.902 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:07:26.920 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:07:28.590 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-21T02:07:28.618 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:07:28.657 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:28.696 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-21T02:07:28.724 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:07:28.763 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:28.926 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-21T02:07:28.954 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:07:28.993 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:29.024 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-21T02:07:29.052 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:07:29.090 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:29.296 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-21T02:07:29.324 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:07:29.354 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:29.410 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-21T02:07:29.438 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:07:29.477 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:29.691 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-21T02:07:29.718 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-21T02:07:29.757 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:07:29.805 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-21T02:07:29.833 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-21T02:07:29.871 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:07:30.085 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-21T02:07:30.113 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:07:30.152 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:07:30.199 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-21T02:07:30.228 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:07:30.266 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:07:30.472 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-21T02:07:30.500 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:30.530 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:30.594 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-21T02:07:30.622 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:30.652 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:30.858 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-21T02:07:30.886 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:30.924 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:30.988 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-21T02:07:31.017 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:31.055 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:31.227 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-21T02:07:31.244 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:31.286 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:31.393 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-21T02:07:31.421 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:31.459 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:31.513 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-21T02:07:31.529 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:07:31.572 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:07:31.678 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-21T02:07:31.695 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:07:31.728 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:07:32.126 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-21T02:07:32.153 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-21T02:07:32.192 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:07:32.232 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-21T02:07:32.260 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-21T02:07:32.299 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:07:32.554 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package modemmanager. 2024-09-21T02:07:32.582 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:07:32.620 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:32.635 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package modemmanager. 2024-09-21T02:07:32.664 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:07:32.702 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:33.309 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package network-manager. 2024-09-21T02:07:33.321 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package network-manager. 2024-09-21T02:07:33.337 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:07:33.350 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:07:33.384 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:07:33.396 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:07:33.946 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ppp. 2024-09-21T02:07:33.966 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ppp. 2024-09-21T02:07:33.974 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-21T02:07:33.994 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-21T02:07:34.237 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:07:34.242 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:07:34.504 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pptp-linux. 2024-09-21T02:07:34.520 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-21T02:07:34.562 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-21T02:07:34.576 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package pptp-linux. 2024-09-21T02:07:34.604 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-21T02:07:34.642 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-21T02:07:34.815 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-21T02:07:34.832 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-21T02:07:34.853 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-21T02:07:34.870 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-21T02:07:34.874 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-21T02:07:34.903 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-21T02:07:35.177 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-21T02:07:35.190 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-21T02:07:35.205 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-21T02:07:35.219 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-21T02:07:35.243 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client-common (238) ... 2024-09-21T02:07:35.257 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-client-common (238) ... 2024-09-21T02:07:35.506 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-21T02:07:35.534 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:35.561 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-21T02:07:35.564 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:35.589 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:35.627 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-common. 2024-09-21T02:07:36.173 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-21T02:07:36.207 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-common. 2024-09-21T02:07:36.230 INFO:teuthology.orchestra.run.smithi122.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-21T02:07:36.235 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-21T02:07:36.258 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-common (238) ... 2024-09-21T02:07:36.292 INFO:teuthology.orchestra.run.smithi096.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-21T02:07:36.322 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-common (238) ... 2024-09-21T02:07:36.765 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-14. 2024-09-21T02:07:36.793 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:36.835 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:36.852 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-14. 2024-09-21T02:07:36.881 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:07:36.923 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:38.297 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql. 2024-09-21T02:07:38.314 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-21T02:07:38.356 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql (14+238) ... 2024-09-21T02:07:38.599 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client. 2024-09-21T02:07:38.616 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-21T02:07:38.658 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client (14+238) ... 2024-09-21T02:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql. 2024-09-21T02:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-21T02:07:38.893 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-21T02:07:38.910 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-21T02:07:38.911 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql (14+238) ... 2024-09-21T02:07:38.952 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-21T02:07:39.130 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-client. 2024-09-21T02:07:39.159 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-21T02:07:39.197 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-client (14+238) ... 2024-09-21T02:07:39.271 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package tcl8.6. 2024-09-21T02:07:39.300 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-21T02:07:39.337 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:07:39.441 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-21T02:07:39.457 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-21T02:07:39.499 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-21T02:07:39.623 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package tcl. 2024-09-21T02:07:39.652 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-21T02:07:39.696 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-21T02:07:39.819 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package tcl8.6. 2024-09-21T02:07:39.847 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-21T02:07:39.885 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:07:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package wpasupplicant. 2024-09-21T02:07:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-21T02:07:40.060 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:07:40.146 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package tcl. 2024-09-21T02:07:40.174 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-21T02:07:40.209 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-21T02:07:40.406 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-21T02:07:40.434 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-21T02:07:40.472 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-21T02:07:40.541 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package wpasupplicant. 2024-09-21T02:07:40.569 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-21T02:07:40.607 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:07:40.850 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-21T02:07:40.879 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-21T02:07:40.926 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:07:40.970 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-21T02:07:40.986 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-21T02:07:41.028 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-21T02:07:41.247 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client-common (238) ... 2024-09-21T02:07:41.415 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-21T02:07:41.443 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-21T02:07:41.465 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-21T02:07:41.490 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:07:41.591 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:07:41.717 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:41.820 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-client-common (238) ... 2024-09-21T02:07:41.851 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:07:41.978 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:07:42.030 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-21T02:07:42.104 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:42.156 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:07:42.299 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:42.425 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:07:42.552 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-21T02:07:42.626 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:07:42.660 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:42.993 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:07:42.993 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:07:43.137 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-21T02:07:43.226 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:07:43.496 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:07:43.622 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:07:43.628 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:07:43.628 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:07:43.739 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:07:43.857 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-21T02:07:44.008 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:07:44.094 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:07:44.134 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:07:44.269 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:07:44.395 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-21T02:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:07:45.008 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:07:45.134 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up dump (0.4b47-3) ... 2024-09-21T02:07:45.228 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-21T02:07:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:45.579 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:07:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up dump (0.4b47-3) ... 2024-09-21T02:07:45.705 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-21T02:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-21T02:07:45.831 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-21T02:07:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:45.957 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:07:45.962 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:07:46.048 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-21T02:07:46.090 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-21T02:07:46.108 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:07:46.209 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:07:46.214 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-21T02:07:46.349 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:07:46.386 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:07:46.440 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-21T02:07:46.509 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:07:46.512 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:46.635 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:07:46.794 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:07:46.895 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:47.013 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-21T02:07:47.021 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-21T02:07:47.181 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-client (14+238) ... 2024-09-21T02:07:47.189 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client (14+238) ... 2024-09-21T02:07:47.315 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:07:47.316 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:07:47.767 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:07:47.768 INFO:teuthology.orchestra.run.smithi122.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-21T02:07:47.768 INFO:teuthology.orchestra.run.smithi122.stdout:which means they are currently configured by ifupdown: 2024-09-21T02:07:47.768 INFO:teuthology.orchestra.run.smithi122.stdout:- enp3s0f1 2024-09-21T02:07:47.768 INFO:teuthology.orchestra.run.smithi122.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-21T02:07:47.768 INFO:teuthology.orchestra.run.smithi122.stdout:remove their configuration from /etc/network/interfaces. 2024-09-21T02:07:47.768 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:07:47.787 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:07:47.787 INFO:teuthology.orchestra.run.smithi096.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-21T02:07:47.787 INFO:teuthology.orchestra.run.smithi096.stdout:which means they are currently configured by ifupdown: 2024-09-21T02:07:47.787 INFO:teuthology.orchestra.run.smithi096.stdout:- enp3s0f1 2024-09-21T02:07:47.787 INFO:teuthology.orchestra.run.smithi096.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-21T02:07:47.788 INFO:teuthology.orchestra.run.smithi096.stdout:remove their configuration from /etc/network/interfaces. 2024-09-21T02:07:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:07:47.891 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-21T02:07:47.918 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-21T02:07:48.223 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-21T02:07:48.228 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-21T02:07:48.533 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-21T02:07:48.542 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-21T02:07:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:49.084 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:07:49.178 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:49.227 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:07:49.304 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-21T02:07:49.328 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-21T02:07:49.430 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-common (238) ... 2024-09-21T02:07:49.437 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-common (238) ... 2024-09-21T02:07:50.315 INFO:teuthology.orchestra.run.smithi122.stdout:Adding user postgres to group ssl-cert 2024-09-21T02:07:50.334 INFO:teuthology.orchestra.run.smithi096.stdout:Adding user postgres to group ssl-cert 2024-09-21T02:07:50.530 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:07:50.531 INFO:teuthology.orchestra.run.smithi122.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-21T02:07:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:07:50.583 INFO:teuthology.orchestra.run.smithi096.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-21T02:07:50.733 INFO:teuthology.orchestra.run.smithi122.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-21T02:07:50.733 INFO:teuthology.orchestra.run.smithi122.stdout:Removing obsolete dictionary files: 2024-09-21T02:07:50.782 INFO:teuthology.orchestra.run.smithi096.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-21T02:07:50.782 INFO:teuthology.orchestra.run.smithi096.stdout:Removing obsolete dictionary files: 2024-09-21T02:07:51.280 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-21T02:07:51.368 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-21T02:07:51.665 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:07:51.982 INFO:teuthology.orchestra.run.smithi122.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-21T02:07:51.999 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-21T02:07:52.040 INFO:teuthology.orchestra.run.smithi122.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-21T02:07:52.040 INFO:teuthology.orchestra.run.smithi122.stdout:This user must also own the server process. 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout:The database cluster will be initialized with locale "en_US". 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout:The default text search configuration will be set to "english". 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout:Data page checksums are disabled. 2024-09-21T02:07:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:07:52.042 INFO:teuthology.orchestra.run.smithi122.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-21T02:07:52.042 INFO:teuthology.orchestra.run.smithi122.stdout:creating subdirectories ... ok 2024-09-21T02:07:52.042 INFO:teuthology.orchestra.run.smithi122.stdout:selecting dynamic shared memory implementation ... posix 2024-09-21T02:07:52.055 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default max_connections ... 100 2024-09-21T02:07:52.074 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default shared_buffers ... 128MB 2024-09-21T02:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-21T02:07:52.084 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default time zone ... Etc/UTC 2024-09-21T02:07:52.085 INFO:teuthology.orchestra.run.smithi122.stdout:creating configuration files ... ok 2024-09-21T02:07:52.114 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-21T02:07:52.167 INFO:teuthology.orchestra.run.smithi096.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-21T02:07:52.168 INFO:teuthology.orchestra.run.smithi096.stdout:This user must also own the server process. 2024-09-21T02:07:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:07:52.168 INFO:teuthology.orchestra.run.smithi096.stdout:The database cluster will be initialized with locale "en_US". 2024-09-21T02:07:52.168 INFO:teuthology.orchestra.run.smithi096.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-21T02:07:52.168 INFO:teuthology.orchestra.run.smithi096.stdout:The default text search configuration will be set to "english". 2024-09-21T02:07:52.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:07:52.169 INFO:teuthology.orchestra.run.smithi096.stdout:Data page checksums are disabled. 2024-09-21T02:07:52.169 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:07:52.169 INFO:teuthology.orchestra.run.smithi096.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-21T02:07:52.169 INFO:teuthology.orchestra.run.smithi096.stdout:creating subdirectories ... ok 2024-09-21T02:07:52.169 INFO:teuthology.orchestra.run.smithi096.stdout:selecting dynamic shared memory implementation ... posix 2024-09-21T02:07:52.182 INFO:teuthology.orchestra.run.smithi096.stdout:selecting default max_connections ... 100 2024-09-21T02:07:52.201 INFO:teuthology.orchestra.run.smithi122.stdout:running bootstrap script ... ok 2024-09-21T02:07:52.203 INFO:teuthology.orchestra.run.smithi096.stdout:selecting default shared_buffers ... 128MB 2024-09-21T02:07:52.213 INFO:teuthology.orchestra.run.smithi096.stdout:selecting default time zone ... Etc/UTC 2024-09-21T02:07:52.214 INFO:teuthology.orchestra.run.smithi096.stdout:creating configuration files ... ok 2024-09-21T02:07:52.328 INFO:teuthology.orchestra.run.smithi096.stdout:running bootstrap script ... ok 2024-09-21T02:07:52.861 INFO:teuthology.orchestra.run.smithi122.stdout:performing post-bootstrap initialization ... ok 2024-09-21T02:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout:performing post-bootstrap initialization ... ok 2024-09-21T02:07:53.538 INFO:teuthology.orchestra.run.smithi122.stdout:syncing data to disk ... ok 2024-09-21T02:07:53.626 INFO:teuthology.orchestra.run.smithi096.stdout:syncing data to disk ... ok 2024-09-21T02:07:53.922 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-21T02:07:54.020 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-21T02:07:56.689 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:56.762 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:56.815 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-21T02:07:56.888 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-21T02:07:56.941 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql (14+238) ... 2024-09-21T02:07:57.022 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql (14+238) ... 2024-09-21T02:07:57.058 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:57.148 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:07:57.178 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:57.274 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:07:57.434 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:07:57.434 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:07:57.530 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:07:57.530 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:07:57.964 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:07:58.065 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-21T02:07:58.110 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:07:58.202 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-21T02:07:58.375 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:07:58.504 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:07:58.527 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:07:58.682 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:07:59.889 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-21T02:07:59.997 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:08:00.051 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-21T02:08:00.151 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:08:00.504 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:08:00.511 DEBUG:teuthology.parallel:result is None 2024-09-21T02:08:00.662 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:08:00.666 DEBUG:teuthology.parallel:result is None 2024-09-21T02:08:00.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:08:00.816 DEBUG:teuthology.orchestra.run.smithi096:> dpkg-query -W -f '${Version}' ceph 2024-09-21T02:08:00.835 INFO:teuthology.orchestra.run.smithi096.stdout:19.3.0-5120-g18432218-1jammy 2024-09-21T02:08:00.835 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:08:00.835 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120-g18432218-1jammy is installed. 2024-09-21T02:08:00.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:08:00.978 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-09-21T02:08:00.998 INFO:teuthology.orchestra.run.smithi122.stdout:19.3.0-5120-g18432218-1jammy 2024-09-21T02:08:00.998 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:08:00.998 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120-g18432218-1jammy is installed. 2024-09-21T02:08:01.000 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T02:08:01.000 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:01.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:08:01.015 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:01.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:08:01.054 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T02:08:01.055 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:01.055 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:08:01.075 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:08:01.131 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:01.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:08:01.143 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:08:01.196 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T02:08:01.197 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:01.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:08:01.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:08:01.267 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:01.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:08:01.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:08:01.333 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T02:08:01.333 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:01.333 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:08:01.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:08:01.402 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:01.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:08:01.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:08:01.469 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T02:08:01.543 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T02:08:01.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-09-21T02:08:01.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-21T02:08:01.559 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T02:08:01.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T02:08:01.600 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T02:08:01.613 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T02:08:01.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T02:08:01.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T02:08:01.678 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T02:08:01.678 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T02:08:01.678 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T02:08:01.678 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:01.678 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T02:08:01.711 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T02:08:01.786 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:01.786 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T02:08:01.792 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T02:08:01.870 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T02:08:01.870 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T02:08:01.870 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/acls}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913407', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWmLErnDFG9xnbGzgX62ASxwVoqJKVXcE/ASwtMkgibIWqpVQ9HWbJrQH4HiewsrxSJxktnQKsuWF1/nFV91kY=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpTc0VuyECkNiSQ9jVWZe5j0kX/uYALbAU8BwMMHubNpeLHk/qVCaaqwrcK3oBS+GMzpwntFdYqevsOFCNT4I8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T02:08:01.871 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T02:08:01.873 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T02:08:01.914 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T02:08:01.920 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T02:08:01.973 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:01.973 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:08:01.978 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:08:01.979 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-21T02:08:02.027 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:08:02.027 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.027 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-09-21T02:08:02.027 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.027 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-21 02:01:46.191472238 +0000 2024-09-21T02:08:02.028 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-21 02:01:45.943474451 +0000 2024-09-21T02:08:02.028 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-21 02:01:45.943474451 +0000 2024-09-21T02:08:02.028 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-21T02:08:02.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:08:02.081 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-21T02:08:02.082 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-21T02:08:02.082 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000373063 s, 1.4 MB/s 2024-09-21T02:08:02.083 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:08:02.132 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-21 02:01:46.671467954 +0000 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-21 02:01:46.423470167 +0000 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-21 02:01:46.423470167 +0000 2024-09-21T02:08:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-21T02:08:02.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:08:02.234 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-21T02:08:02.234 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-21T02:08:02.234 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000421776 s, 1.2 MB/s 2024-09-21T02:08:02.235 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:08:02.284 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-21 02:01:47.151463670 +0000 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-21 02:01:46.895465955 +0000 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-21 02:01:46.895465955 +0000 2024-09-21T02:08:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-21T02:08:02.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:08:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-21T02:08:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-21T02:08:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000424547 s, 1.2 MB/s 2024-09-21T02:08:02.387 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:08:02.438 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-21 02:01:47.631459385 +0000 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-21 02:01:47.395461492 +0000 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-21 02:01:47.395461492 +0000 2024-09-21T02:08:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-21T02:08:02.484 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:08:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-21T02:08:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-21T02:08:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000345047 s, 1.5 MB/s 2024-09-21T02:08:02.539 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:08:02.588 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:02.588 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:02.588 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:08:02.594 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:08:02.594 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-21T02:08:02.642 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:08:02.642 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.642 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-21T02:08:02.642 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.642 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 02:01:47.346712613 +0000 2024-09-21T02:08:02.642 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 02:01:47.342712649 +0000 2024-09-21T02:08:02.643 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 02:01:47.342712649 +0000 2024-09-21T02:08:02.643 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-21T02:08:02.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:08:02.696 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T02:08:02.697 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T02:08:02.697 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00044976 s, 1.1 MB/s 2024-09-21T02:08:02.698 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:08:02.747 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-21T02:08:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:08:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.798 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-09-21T02:08:02.798 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.798 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 02:01:48.070706147 +0000 2024-09-21T02:08:02.798 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 02:01:47.846708148 +0000 2024-09-21T02:08:02.798 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 02:01:47.846708148 +0000 2024-09-21T02:08:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-21T02:08:02.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:08:02.853 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T02:08:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T02:08:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000320388 s, 1.6 MB/s 2024-09-21T02:08:02.855 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:08:02.903 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-21T02:08:02.949 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:08:02.949 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:02.949 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-21T02:08:02.950 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:02.950 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 02:01:48.550701859 +0000 2024-09-21T02:08:02.950 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 02:01:48.278704289 +0000 2024-09-21T02:08:02.950 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 02:01:48.278704289 +0000 2024-09-21T02:08:02.950 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-21T02:08:02.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:08:03.004 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T02:08:03.004 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T02:08:03.004 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000431702 s, 1.2 MB/s 2024-09-21T02:08:03.005 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:08:03.057 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 02:01:49.094697000 +0000 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 02:01:48.818699465 +0000 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 02:01:48.818699465 +0000 2024-09-21T02:08:03.106 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-21T02:08:03.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:08:03.160 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T02:08:03.160 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T02:08:03.160 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00037936 s, 1.3 MB/s 2024-09-21T02:08:03.162 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:08:03.211 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:03.211 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T02:08:03.211 INFO:tasks.ceph:Generating config... 2024-09-21T02:08:03.212 INFO:tasks.ceph:[client] pid file = 2024-09-21T02:08:03.212 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T02:08:03.213 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T02:08:03.213 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T02:08:03.213 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T02:08:03.213 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T02:08:03.213 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T02:08:03.213 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T02:08:03.213 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T02:08:03.213 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-21T02:08:03.213 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-21T02:08:03.213 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T02:08:03.213 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T02:08:03.214 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-21T02:08:03.214 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T02:08:03.214 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T02:08:03.214 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T02:08:03.214 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T02:08:03.214 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T02:08:03.214 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T02:08:03.214 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T02:08:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T02:08:03.293 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T02:08:03.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:08:03.369 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.96'), ('mon.b', '172.21.15.122'), ('mon.c', '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]')] 2024-09-21T02:08:03.369 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.96,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T02:08:03.370 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:03.370 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T02:08:03.414 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.96 --add b 172.21.15.122 --addv c '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:03.527 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:03.527 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: generated fsid 892bf3fa-0a6f-4242-b0ca-536a4d478630 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 0 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 892bf3fa-0a6f-4242-b0ca-536a4d478630 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:last_changed 2024-09-21T02:08:03.526770+0000 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-09-21T02:08:03.526770+0000 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:min_mon_release 17 (quincy) 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:election_strategy: 1 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.b 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:2: [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] mon.c 2024-09-21T02:08:03.528 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T02:08:03.533 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T02:08:03.538 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 892bf3fa-0a6f-4242-b0ca-536a4d478630... 2024-09-21T02:08:03.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T02:08:03.584 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T02:08:03.608 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-09-21T02:08:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-09-21T02:08:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.96,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]" 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 892bf3fa-0a6f-4242-b0ca-536a4d478630 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug trim objects = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-09-21T02:08:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 20 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 20 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-09-21T02:08:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 10 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:08:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:08:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: mon down mkfs grace = 300 2024-09-21T02:08:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = "" 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2024-09-21T02:08:03.618 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d] 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-21T02:08:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T02:08:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.96,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]" 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 892bf3fa-0a6f-4242-b0ca-536a4d478630 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-21T02:08:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-21T02:08:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 20 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 20 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 10 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-21T02:08:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-21T02:08:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T02:08:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: mon down mkfs grace = 300 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T02:08:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = "" 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-09-21T02:08:03.629 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-09-21T02:08:03.630 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T02:08:03.631 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T02:08:03.755 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T02:08:03.755 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:03.755 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T02:08:03.760 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:03.760 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T02:08:03.807 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:08:03.807 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:03.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T02:08:03.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:08:03.871 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:03.872 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:03.919 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:08:03.919 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:03.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T02:08:03.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:08:03.942 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:03.943 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:03.990 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T02:08:03.990 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T02:08:03.990 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T02:08:04.064 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T02:08:04.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T02:08:04.144 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T02:08:04.158 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T02:08:04.159 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T02:08:04.229 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T02:08:04.236 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T02:08:04.248 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T02:08:04.357 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T02:08:04.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T02:08:04.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T02:08:04.446 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T02:08:04.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T02:08:04.465 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T02:08:04.583 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T02:08:04.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T02:08:04.601 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T02:08:04.601 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T02:08:04.666 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T02:08:04.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T02:08:04.736 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T02:08:04.749 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T02:08:04.817 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T02:08:04.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T02:08:04.895 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T02:08:04.909 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T02:08:04.909 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T02:08:04.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T02:08:04.923 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:04.923 INFO:tasks.ceph:role: osd.0 2024-09-21T02:08:04.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:08:04.923 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T02:08:04.986 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:04.987 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:05.464 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-21T02:08:05.466 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-21T02:08:05.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T02:08:05.624 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T02:08:05.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:05.635 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:05.636 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T02:08:05.695 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:05.695 INFO:tasks.ceph:role: osd.1 2024-09-21T02:08:05.695 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:08:05.695 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:05.764 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:06.212 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-21T02:08:06.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-21T02:08:06.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T02:08:06.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T02:08:06.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:06.329 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:06.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T02:08:06.382 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:06.382 INFO:tasks.ceph:role: osd.2 2024-09-21T02:08:06.382 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:08:06.382 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T02:08:06.442 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:06.442 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:06.443 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:06.443 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:06.443 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:06.898 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-21T02:08:06.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-21T02:08:06.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T02:08:07.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T02:08:07.018 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:07.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:07.019 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T02:08:07.075 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:07.075 INFO:tasks.ceph:role: osd.3 2024-09-21T02:08:07.075 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:08:07.075 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T02:08:07.139 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:07.140 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:07.647 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-21T02:08:07.649 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-21T02:08:07.649 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T02:08:07.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T02:08:07.817 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:07.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:07.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:07.964+0000 7f9a5bc987c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T02:08:07.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:07.968+0000 7f9a5bc987c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T02:08:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:07.968+0000 7f9a5bc987c0 -1 bdev(0x55f55e62aa00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:07.968+0000 7f9a5bc987c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T02:08:11.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T02:08:11.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:11.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:11.212+0000 7f02cdea37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T02:08:11.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:11.212+0000 7f02cdea37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T02:08:11.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:11.212+0000 7f02cdea37c0 -1 bdev(0x561b319dea00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:11.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:11.212+0000 7f02cdea37c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T02:08:14.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T02:08:14.351 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:14.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:14.456+0000 7f3d88bd37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T02:08:14.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:14.456+0000 7f3d88bd37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T02:08:14.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:14.456+0000 7f3d88bd37c0 -1 bdev(0x559ff734ca00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:14.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:14.456+0000 7f3d88bd37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T02:08:17.593 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T02:08:17.606 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:17.708+0000 7f09a31b47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T02:08:17.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:17.708+0000 7f09a31b47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T02:08:17.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:17.708+0000 7f09a31b47c0 -1 bdev(0x55f3ccdfea00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:17.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:17.708+0000 7f09a31b47c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T02:08:20.874 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T02:08:20.887 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T02:08:20.902 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:20.902 INFO:tasks.ceph:role: osd.4 2024-09-21T02:08:20.902 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:08:20.902 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:20.965 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:20.966 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:21.419 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T02:08:21.421 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T02:08:21.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T02:08:21.528 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T02:08:21.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:21.538 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:21.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T02:08:21.591 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:21.591 INFO:tasks.ceph:role: osd.5 2024-09-21T02:08:21.591 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:08:21.592 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T02:08:21.654 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:21.654 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:21.655 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:21.655 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:21.655 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:22.112 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T02:08:22.115 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T02:08:22.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T02:08:22.230 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T02:08:22.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:22.242 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:22.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T02:08:22.296 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:22.296 INFO:tasks.ceph:role: osd.6 2024-09-21T02:08:22.296 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:08:22.297 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T02:08:22.357 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:22.357 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:22.357 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:22.357 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:22.357 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:22.358 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:22.358 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:22.358 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:22.358 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:22.358 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:22.802 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T02:08:22.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T02:08:22.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T02:08:22.937 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T02:08:22.948 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:22.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:22.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T02:08:23.006 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:08:23.006 INFO:tasks.ceph:role: osd.7 2024-09-21T02:08:23.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:08:23.006 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T02:08:23.073 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:08:23.074 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:08:23.515 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T02:08:23.517 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T02:08:23.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T02:08:23.610 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T02:08:23.619 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:08:23.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:08:23.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:23.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:23.759+0000 7f9cb34d67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T02:08:23.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:23.759+0000 7f9cb34d67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T02:08:23.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:23.759+0000 7f9cb34d67c0 -1 bdev(0x562d8a4baa00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:23.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:23.763+0000 7f9cb34d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T02:08:26.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T02:08:26.894 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:27.003+0000 7fd9e830d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T02:08:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:27.003+0000 7fd9e830d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T02:08:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:27.003+0000 7fd9e830d7c0 -1 bdev(0x55d4f0b84a00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:27.003+0000 7fd9e830d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T02:08:30.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T02:08:30.189 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:30.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:30.299+0000 7fae965be7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T02:08:30.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:30.299+0000 7fae965be7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T02:08:30.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:30.299+0000 7fae965be7c0 -1 bdev(0x55623dfe0a00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:30.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:30.299+0000 7fae965be7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T02:08:31.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:08:31.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:08:33.422 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T02:08:33.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:33.543+0000 7f56b2ec87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T02:08:33.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:33.543+0000 7f56b2ec87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T02:08:33.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:33.543+0000 7f56b2ec87c0 -1 bdev(0x565331b62a00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T02:08:33.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:33.543+0000 7f56b2ec87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T02:08:36.676 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T02:08:36.687 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T02:08:36.687 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T02:08:36.702 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.702 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T02:08:36.754 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.754 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T02:08:36.805 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.806 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T02:08:36.858 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T02:08:36.917 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T02:08:36.970 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:36.970 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T02:08:37.027 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T02:08:37.040 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.041 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T02:08:37.096 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.096 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T02:08:37.152 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T02:08:37.208 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T02:08:37.261 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T02:08:37.316 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.316 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T02:08:37.368 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:37.368 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T02:08:37.374 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:37.374 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T02:08:37.419 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.419 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T02:08:37.424 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:37.424 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T02:08:37.474 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T02:08:37.474 DEBUG:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T02:08:37.476 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T02:08:37.519 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.y] 2024-09-21T02:08:37.519 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mC8rIAxAAcgR6MpZGGj2MxGPuYVtgbw== 2024-09-21T02:08:37.519 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2024-09-21T02:08:37.519 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mkCWlDRAAzdu0wONk/0Lupo5cZ3NHzQ== 2024-09-21T02:08:37.519 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2024-09-21T02:08:37.519 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mO6U8FRAAXyQxOIliLEhF5gm7p8ZHmw== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCHKu5mu92zORAAaLLvKlmkKyWEyhVMFOUsgQ== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCLKu5mqWPTDBAA6vUHoA/PGj0RhabCpEzrHQ== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCOKu5mkhNlGxAA3iSsM1nh92w+bNjtOGrK5g== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCRKu5mUT5xKhAAu1+5N4qMqW7sUyyk5SmOPw== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mLC2wCBAAZw8at6A2SvR34GvkK2hEUQ== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mnXSzGhAA0dpvCpA6xxnAsiJJ20Y1Qg== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d] 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mMpzaIhAAY0BZMv3gutB1Aqe1bPOGgQ== 2024-09-21T02:08:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCXKu5mVAaYLRAA07ptXx3hhzgxcqbcFPe5XQ== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCbKu5mBfaZABAA0RK0Pv4EC8imtRDSkCevKg== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.6] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCeKu5m1+02EhAAiWYPYmpourCdhLuJHMNoRQ== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.7] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQChKu5mfXOxIBAAz1mbwiOSpnL+a+zCa4krBA== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5my+CoJxAAog4sBSDU0v9/F60t9/U1nQ== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[client.2] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mZqfSKxAAaIcNdXRvmymRHPV2adIDyg== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5mojXOMBAAy74zTgdiJ5bDnXgc9VvuNg== 2024-09-21T02:08:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:[client.3] 2024-09-21T02:08:37.522 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5m8Ud8NRAAhHIgwf6D9G3sWEOKFyKVmA== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.y] 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mC8rIAxAAcgR6MpZGGj2MxGPuYVtgbw== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mkCWlDRAAzdu0wONk/0Lupo5cZ3NHzQ== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mO6U8FRAAXyQxOIliLEhF5gm7p8ZHmw== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCHKu5mu92zORAAaLLvKlmkKyWEyhVMFOUsgQ== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCLKu5mqWPTDBAA6vUHoA/PGj0RhabCpEzrHQ== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCOKu5mkhNlGxAA3iSsM1nh92w+bNjtOGrK5g== 2024-09-21T02:08:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCRKu5mUT5xKhAAu1+5N4qMqW7sUyyk5SmOPw== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mLC2wCBAAZw8at6A2SvR34GvkK2hEUQ== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mnXSzGhAA0dpvCpA6xxnAsiJJ20Y1Qg== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mMpzaIhAAY0BZMv3gutB1Aqe1bPOGgQ== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCXKu5mVAaYLRAA07ptXx3hhzgxcqbcFPe5XQ== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCbKu5mBfaZABAA0RK0Pv4EC8imtRDSkCevKg== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.6] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCeKu5m1+02EhAAiWYPYmpourCdhLuJHMNoRQ== 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.7] 2024-09-21T02:08:37.529 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQChKu5mfXOxIBAAz1mbwiOSpnL+a+zCa4krBA== 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5my+CoJxAAog4sBSDU0v9/F60t9/U1nQ== 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout:[client.2] 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mZqfSKxAAaIcNdXRvmymRHPV2adIDyg== 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout:[client.1] 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5mojXOMBAAy74zTgdiJ5bDnXgc9VvuNg== 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout:[client.3] 2024-09-21T02:08:37.530 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEKu5m8Ud8NRAAhHIgwf6D9G3sWEOKFyKVmA== 2024-09-21T02:08:37.531 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:08:37.568 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:08:37.654 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:37.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:37.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:37.736 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:37.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:37.809 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:37.883 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:37.885 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:37.957 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:37.960 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:08:38.108 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:08:38.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:38.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:38.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:38.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:08:38.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.328 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.476 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.547 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:08:38.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.617 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.690 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.764 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.834 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:08:38.904 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T02:08:38.904 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T02:08:38.915 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:08:39.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T02:08:39.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T02:08:39.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:08:39.180 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T02:08:39.190 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T02:08:39.242 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:08:39.356 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T02:08:39.365 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:39.368 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:08:39.412 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T02:08:39.412 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T02:08:39.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T02:08:39.416 INFO:tasks.ceph.mon.a:Started 2024-09-21T02:08:39.416 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T02:08:39.416 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T02:08:39.454 INFO:tasks.ceph.mon.b:Started 2024-09-21T02:08:39.454 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T02:08:39.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T02:08:39.456 INFO:tasks.ceph.mon.c:Started 2024-09-21T02:08:39.456 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T02:08:39.456 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T02:08:39.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T02:08:39.459 INFO:tasks.ceph.mgr.y:Started 2024-09-21T02:08:39.459 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T02:08:39.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T02:08:39.460 INFO:tasks.ceph.mgr.x:Started 2024-09-21T02:08:39.461 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T02:08:39.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T02:08:39.463 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T02:08:39.463 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T02:08:39.507 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T02:08:39.507 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T02:08:39.555 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T02:08:39.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T02:08:39.603 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T02:08:39.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T02:08:39.647 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T02:08:39.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T02:08:39.691 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T02:08:39.691 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T02:08:39.735 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T02:08:39.735 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T02:08:39.779 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T02:08:39.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T02:08:39.823 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T02:08:39.823 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T02:08:39.867 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T02:08:39.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T02:08:39.911 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T02:08:39.911 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T02:08:39.955 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T02:08:39.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T02:08:40.003 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T02:08:40.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T02:08:40.047 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T02:08:40.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T02:08:40.091 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T02:08:40.091 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T02:08:40.139 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T02:08:40.139 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T02:08:40.183 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T02:08:40.183 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T02:08:40.227 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T02:08:40.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T02:08:40.275 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T02:08:40.275 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T02:08:40.319 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T02:08:40.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T02:08:40.363 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T02:08:40.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T02:08:40.407 DEBUG:tasks.ceph:set 22 configs 2024-09-21T02:08:40.407 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.360 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.360 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 -- 172.21.15.96:0/435596103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b74007de0 msgr2=0x7f1b901f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:45.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 --2- 172.21.15.96:0/435596103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b74007de0 0x7f1b901f2f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b8001be20 tx=0x7f1b800b06b0 comp rx=0 tx=0).stop 2024-09-21T02:08:45.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 -- 172.21.15.96:0/435596103 shutdown_connections 2024-09-21T02:08:45.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 --2- 172.21.15.96:0/435596103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b74008730 0x7f1b901f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:45.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 --2- 172.21.15.96:0/435596103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b74007de0 0x7f1b901f2f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:45.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 --2- 172.21.15.96:0/435596103 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b74003bc0 0x7f1b901579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:45.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 -- 172.21.15.96:0/435596103 >> 172.21.15.96:0/435596103 conn(0x7f1b900cd6a0 msgr2=0x7f1b9015ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:45.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 -- 172.21.15.96:0/435596103 shutdown_connections 2024-09-21T02:08:45.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:45.784+0000 7f1b9833d640 1 -- 172.21.15.96:0/435596103 wait complete. 2024-09-21T02:08:45.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi096') 2024-09-21T02:08:45.813 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.004+0000 7ffac2102640 1 -- 172.21.15.96:0/214320143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8006610 msgr2=0x7ffaa8009310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.004+0000 7ffac2102640 1 --2- 172.21.15.96:0/214320143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8006610 0x7ffaa8009310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006100 tx=0x7ffaac01e3b0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.008+0000 7ffac2102640 1 -- 172.21.15.96:0/214320143 shutdown_connections 2024-09-21T02:08:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.008+0000 7ffac2102640 1 --2- 172.21.15.96:0/214320143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffaa800af70 0x7ffaa800d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.008+0000 7ffac2102640 1 --2- 172.21.15.96:0/214320143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8006610 0x7ffaa8009310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.008+0000 7ffac2102640 1 -- 172.21.15.96:0/214320143 >> 172.21.15.96:0/214320143 conn(0x7ffabc0cd6a0 msgr2=0x7ffabc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.008+0000 7ffac2102640 1 -- 172.21.15.96:0/214320143 shutdown_connections 2024-09-21T02:08:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.008+0000 7ffac2102640 1 -- 172.21.15.96:0/214320143 wait complete. 2024-09-21T02:08:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.012+0000 7ffac2102640 1 Processor -- start 2024-09-21T02:08:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.012+0000 7ffac2102640 1 -- start start 2024-09-21T02:08:46.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 0x7ffabc1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa8006610 0x7ffabc1545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffaa800af70 0x7ffabc159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc0cf3d0 con 0x7ffaa8003bb0 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc0cf250 con 0x7ffaa8006610 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ffabc0cf550 con 0x7ffaa800af70 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 0x7ffabc1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 0x7ffabc1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46692/0 (socket says 172.21.15.96:46692) 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 -- 172.21.15.96:0/2253031899 learned_addr learned my addr 172.21.15.96:0/2253031899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabbfff640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffaa800af70 0x7ffabc159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabaffd640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa8006610 0x7ffabc1545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 -- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffaa800af70 msgr2=0x7ffabc159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffaa800af70 0x7ffabc159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 -- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa8006610 msgr2=0x7ffabc1545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa8006610 0x7ffabc1545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 -- 172.21.15.96:0/2253031899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc155040 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffabb7fe640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 0x7ffabc1525c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0b1040 tx=0x7ffaac0ae9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0be070 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaac0aebd0 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0b3040 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc155320 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc152b00 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffaac0c1070 con 0x7ffaa8003bb0 2024-09-21T02:08:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffaac0c3980 con 0x7ffaa8003bb0 2024-09-21T02:08:46.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.016+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc0cfab0 con 0x7ffaa8003bb0 2024-09-21T02:08:46.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.020+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffaac0da860 con 0x7ffaa8003bb0 2024-09-21T02:08:46.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.076+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7ffabc0cdb50 con 0x7ffaa8003bb0 2024-09-21T02:08:46.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffaa3fff640 1 -- 172.21.15.96:0/2253031899 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v21)=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7ffaac0c8030 con 0x7ffaa8003bb0 2024-09-21T02:08:46.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 msgr2=0x7ffabc1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 0x7ffabc1525c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0b1040 tx=0x7ffaac0ae9e0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 shutdown_connections 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffaa800af70 0x7ffabc159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa8006610 0x7ffabc1545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 --2- 172.21.15.96:0/2253031899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa8003bb0 0x7ffabc1525c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 >> 172.21.15.96:0/2253031899 conn(0x7ffabc0cd6a0 msgr2=0x7ffabc06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 shutdown_connections 2024-09-21T02:08:46.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.084+0000 7ffac2102640 1 -- 172.21.15.96:0/2253031899 wait complete. 2024-09-21T02:08:46.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.107 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.107 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.364+0000 7fb211a99640 1 -- 172.21.15.96:0/1385376668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f80088c0 msgr2=0x7fb1f800d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.364+0000 7fb211a99640 1 --2- 172.21.15.96:0/1385376668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f80088c0 0x7fb1f800d3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb20c073c50 tx=0x7fb20401d800 comp rx=0 tx=0).stop 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.364+0000 7fb211a99640 1 -- 172.21.15.96:0/1385376668 shutdown_connections 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.364+0000 7fb211a99640 1 --2- 172.21.15.96:0/1385376668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f80088c0 0x7fb1f800d3e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.364+0000 7fb211a99640 1 --2- 172.21.15.96:0/1385376668 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1f8007da0 0x7fb1f80081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.364+0000 7fb211a99640 1 -- 172.21.15.96:0/1385376668 >> 172.21.15.96:0/1385376668 conn(0x7fb20c0d1130 msgr2=0x7fb20c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.368+0000 7fb211a99640 1 -- 172.21.15.96:0/1385376668 shutdown_connections 2024-09-21T02:08:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.368+0000 7fb211a99640 1 -- 172.21.15.96:0/1385376668 wait complete. 2024-09-21T02:08:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.372+0000 7fb211a99640 1 Processor -- start 2024-09-21T02:08:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.372+0000 7fb211a99640 1 -- start start 2024-09-21T02:08:46.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1f8003b30 0x7fb20c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f8007da0 0x7fb20c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f80088c0 0x7fb20c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c0cdea0 con 0x7fb1f80088c0 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c0cdd20 con 0x7fb1f8007da0 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb20c0ce020 con 0x7fb1f8003b30 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f80088c0 0x7fb20c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f8007da0 0x7fb20c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f8007da0 0x7fb20c14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.96:57184/0 (socket says 172.21.15.96:57184) 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20affd640 1 -- 172.21.15.96:0/2832113488 learned_addr learned my addr 172.21.15.96:0/2832113488 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20b7fe640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1f8003b30 0x7fb20c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 -- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1f8003b30 msgr2=0x7fb20c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1f8003b30 0x7fb20c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 -- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f8007da0 msgr2=0x7fb20c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f8007da0 0x7fb20c14ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 -- 172.21.15.96:0/2832113488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c157c80 con 0x7fb1f80088c0 2024-09-21T02:08:46.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb20bfff640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f80088c0 0x7fb20c159170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb204010000 tx=0x7fb204003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:46.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2040110d0 con 0x7fb1f80088c0 2024-09-21T02:08:46.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c157f60 con 0x7fb1f80088c0 2024-09-21T02:08:46.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb204004560 con 0x7fb1f80088c0 2024-09-21T02:08:46.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2040a6570 con 0x7fb1f80088c0 2024-09-21T02:08:46.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c158520 con 0x7fb1f80088c0 2024-09-21T02:08:46.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2040be060 con 0x7fb1f80088c0 2024-09-21T02:08:46.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb2040c0850 con 0x7fb1f80088c0 2024-09-21T02:08:46.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.376+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c0713d0 con 0x7fb1f80088c0 2024-09-21T02:08:46.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.380+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb204017580 con 0x7fb1f80088c0 2024-09-21T02:08:46.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.436+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fb20c15ecd0 con 0x7fb1f80088c0 2024-09-21T02:08:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb1f3fff640 1 -- 172.21.15.96:0/2832113488 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v22)=0 v22) ==== 108+0+0 (secure 0 0 0) 0x7fb2040a6b40 con 0x7fb1f80088c0 2024-09-21T02:08:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f80088c0 msgr2=0x7fb20c159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f80088c0 0x7fb20c159170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb204010000 tx=0x7fb204003ed0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 shutdown_connections 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f80088c0 0x7fb20c159170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f8007da0 0x7fb20c14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 --2- 172.21.15.96:0/2832113488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1f8003b30 0x7fb20c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 >> 172.21.15.96:0/2832113488 conn(0x7fb20c0d1130 msgr2=0x7fb20c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 shutdown_connections 2024-09-21T02:08:46.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.440+0000 7fb211a99640 1 -- 172.21.15.96:0/2832113488 wait complete. 2024-09-21T02:08:46.464 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.464 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.464 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 -- 172.21.15.96:0/356268339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 msgr2=0x7ffa74006610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 --2- 172.21.15.96:0/356268339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 0x7ffa74006610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c006b30 tx=0x7ffa7c01d770 comp rx=0 tx=0).stop 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 -- 172.21.15.96:0/356268339 shutdown_connections 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 --2- 172.21.15.96:0/356268339 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74006b50 0x7ffa74006f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 --2- 172.21.15.96:0/356268339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 0x7ffa74006610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 -- 172.21.15.96:0/356268339 >> 172.21.15.96:0/356268339 conn(0x7ffa900cf150 msgr2=0x7ffa900cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 -- 172.21.15.96:0/356268339 shutdown_connections 2024-09-21T02:08:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.496+0000 7ffa97031640 1 -- 172.21.15.96:0/356268339 wait complete. 2024-09-21T02:08:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 Processor -- start 2024-09-21T02:08:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 -- start start 2024-09-21T02:08:46.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 0x7ffa900c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 0x7ffa900b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74007670 0x7ffa90157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa900cca70 con 0x7ffa74006b50 2024-09-21T02:08:46.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa900b2b60 con 0x7ffa74003ae0 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa97031640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ffa900ccbf0 con 0x7ffa74007670 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 0x7ffa900b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 0x7ffa900b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46706/0 (socket says 172.21.15.96:46706) 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 -- 172.21.15.96:0/2043431744 learned_addr learned my addr 172.21.15.96:0/2043431744 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa94dcd640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 0x7ffa900c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa955ce640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74007670 0x7ffa90157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 -- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74007670 msgr2=0x7ffa90157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74007670 0x7ffa90157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 -- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 msgr2=0x7ffa900c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 0x7ffa900c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.500+0000 7ffa8ffff640 1 -- 172.21.15.96:0/2043431744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa900d0860 con 0x7ffa74006b50 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8ffff640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 0x7ffa900b7980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005cb0 tx=0x7ffa84011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84023030 con 0x7ffa74006b50 2024-09-21T02:08:46.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa84010e90 con 0x7ffa74006b50 2024-09-21T02:08:46.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa90161b50 con 0x7ffa74006b50 2024-09-21T02:08:46.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84005240 con 0x7ffa74006b50 2024-09-21T02:08:46.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa90162110 con 0x7ffa74006b50 2024-09-21T02:08:46.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa84026070 con 0x7ffa74006b50 2024-09-21T02:08:46.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa8401fa40 con 0x7ffa74006b50 2024-09-21T02:08:46.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa90162310 con 0x7ffa74006b50 2024-09-21T02:08:46.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.504+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffa90162310 con 0x7ffa74006b50 2024-09-21T02:08:46.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.560+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7ffa900b4310 con 0x7ffa74006b50 2024-09-21T02:08:46.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.564+0000 7ffa8d7fa640 1 -- 172.21.15.96:0/2043431744 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ffa84005530 con 0x7ffa74006b50 2024-09-21T02:08:46.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 msgr2=0x7ffa900b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 0x7ffa900b7980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa84005cb0 tx=0x7ffa84011fd0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 shutdown_connections 2024-09-21T02:08:46.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74007670 0x7ffa90157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa74006b50 0x7ffa900b7980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 --2- 172.21.15.96:0/2043431744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa74003ae0 0x7ffa900c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 >> 172.21.15.96:0/2043431744 conn(0x7ffa900cf150 msgr2=0x7ffa900701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 shutdown_connections 2024-09-21T02:08:46.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.568+0000 7ffa97031640 1 -- 172.21.15.96:0/2043431744 wait complete. 2024-09-21T02:08:46.589 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi096') 2024-09-21T02:08:46.589 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config dump 2024-09-21T02:08:46.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 -- 172.21.15.96:0/2923158654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 msgr2=0x7f5694003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 --2- 172.21.15.96:0/2923158654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f5694003ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56a40016f0 tx=0x7f56a401c5b0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 -- 172.21.15.96:0/2923158654 shutdown_connections 2024-09-21T02:08:46.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 --2- 172.21.15.96:0/2923158654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f5694003ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 -- 172.21.15.96:0/2923158654 >> 172.21.15.96:0/2923158654 conn(0x7f56b40d1130 msgr2=0x7f56b40d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 -- 172.21.15.96:0/2923158654 shutdown_connections 2024-09-21T02:08:46.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.768+0000 7f56b8f36640 1 -- 172.21.15.96:0/2923158654 wait complete. 2024-09-21T02:08:46.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 Processor -- start 2024-09-21T02:08:46.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 -- start start 2024-09-21T02:08:46.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f56b4157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b4159010 0x7f56b41519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b41f9530 0x7f56b41f9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4160d40 con 0x7f5694003a60 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4160bc0 con 0x7f56b41f9530 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f56b4160ec0 con 0x7f56b4159010 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f56b4157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f56b4157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46712/0 (socket says 172.21.15.96:46712) 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 -- 172.21.15.96:0/604264638 learned_addr learned my addr 172.21.15.96:0/604264638 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b2d9d640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b41f9530 0x7f56b41f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56abfff640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b4159010 0x7f56b41519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 -- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b4159010 msgr2=0x7f56b41519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b4159010 0x7f56b41519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 -- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b41f9530 msgr2=0x7f56b41f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b41f9530 0x7f56b41f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 -- 172.21.15.96:0/604264638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b4151f00 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b259c640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f56b4157e70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f56a401acb0 tx=0x7f56a4006500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003600 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56a40b3040 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41fca60 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003ab0 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41fd020 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56a40bf060 con 0x7f5694003a60 2024-09-21T02:08:46.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.772+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f56a40c1790 con 0x7f5694003a60 2024-09-21T02:08:46.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.776+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b406edb0 con 0x7f5694003a60 2024-09-21T02:08:46.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.776+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f56a40c6020 con 0x7f5694003a60 2024-09-21T02:08:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f56b4152ce0 con 0x7f5694003a60 2024-09-21T02:08:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56aaffd640 1 -- 172.21.15.96:0/604264638 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f56b4071b60 con 0x7f5694003a60 2024-09-21T02:08:46.839 INFO:teuthology.orchestra.run.smithi096.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:08:46.839 INFO:teuthology.orchestra.run.smithi096.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced debug_client 20/20 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced debug_mds 20/20 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced debug_ms 2/2 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds dev mds_debug_frag true 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds dev mds_verify_scatter true 2024-09-21T02:08:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced client_mount_timeout 600 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced debug_client 20/20 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced debug_ms 1/1 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:08:46.841 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 msgr2=0x7f56b4157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f56b4157e70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f56a401acb0 tx=0x7f56a4006500 comp rx=0 tx=0).stop 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 shutdown_connections 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b41f9530 0x7f56b41f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b4159010 0x7f56b41519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 --2- 172.21.15.96:0/604264638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5694003a60 0x7f56b4157e70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 >> 172.21.15.96:0/604264638 conn(0x7f56b40d1130 msgr2=0x7f56b40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 shutdown_connections 2024-09-21T02:08:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:46.836+0000 7f56b8f36640 1 -- 172.21.15.96:0/604264638 wait complete. 2024-09-21T02:08:46.858 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T02:08:46.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T02:08:47.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 -- 172.21.15.96:0/1221458502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 msgr2=0x7f5f0c15b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1221458502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c15b0f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c073b00 tx=0x7f5efc01c800 comp rx=0 tx=0).stop 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 -- 172.21.15.96:0/1221458502 shutdown_connections 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1221458502 >> v2:172.21.15.122:3300/0 conn(0x7f5f0c15ed80 0x7f5f0c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1221458502 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f0c071210 0x7f5f0c071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1221458502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c15b0f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 -- 172.21.15.96:0/1221458502 >> 172.21.15.96:0/1221458502 conn(0x7f5f0c06fdb0 msgr2=0x7f5f0c0701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 -- 172.21.15.96:0/1221458502 shutdown_connections 2024-09-21T02:08:47.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 -- 172.21.15.96:0/1221458502 wait complete. 2024-09-21T02:08:47.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 Processor -- start 2024-09-21T02:08:47.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.000+0000 7f5f11a36640 1 -- start start 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f0c071210 0x7f5f0c1633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f0c15ed80 0x7f5f0c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c06da30 con 0x7f5f0c072210 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c06d8b0 con 0x7f5f0c15ed80 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c06dbb0 con 0x7f5f0c071210 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c163910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58082/0 (socket says 172.21.15.96:58082) 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 -- 172.21.15.96:0/1422354415 learned_addr learned my addr 172.21.15.96:0/1422354415 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0affd640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f0c071210 0x7f5f0c1633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0b7fe640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f0c15ed80 0x7f5f0c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 -- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f0c071210 msgr2=0x7f5f0c1633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f0c071210 0x7f5f0c1633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 -- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f0c15ed80 msgr2=0x7f5f0c153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f0c15ed80 0x7f5f0c153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 -- 172.21.15.96:0/1422354415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c165c00 con 0x7f5f0c072210 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0b7fe640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f0c15ed80 0x7f5f0c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f0a7fc640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c163910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000b440 tx=0x7f5f0000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00015020 con 0x7f5f0c072210 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f000119a0 con 0x7f5f0c072210 2024-09-21T02:08:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c165640 con 0x7f5f0c072210 2024-09-21T02:08:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00011df0 con 0x7f5f0c072210 2024-09-21T02:08:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c156a80 con 0x7f5f0c072210 2024-09-21T02:08:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f00018020 con 0x7f5f0c072210 2024-09-21T02:08:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f000109b0 con 0x7f5f0c072210 2024-09-21T02:08:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.004+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c1fa730 con 0x7f5f0c072210 2024-09-21T02:08:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.008+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5f0001f020 con 0x7f5f0c072210 2024-09-21T02:08:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.068+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5f0c15b0f0 con 0x7f5f0c072210 2024-09-21T02:08:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5ef3fff640 1 -- 172.21.15.96:0/1422354415 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f5f0000d040 con 0x7f5f0c072210 2024-09-21T02:08:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 msgr2=0x7f5f0c163910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c163910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000b440 tx=0x7f5f0000b900 comp rx=0 tx=0).stop 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 shutdown_connections 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f0c15ed80 0x7f5f0c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f0c072210 0x7f5f0c163910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 --2- 172.21.15.96:0/1422354415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f0c071210 0x7f5f0c1633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 >> 172.21.15.96:0/1422354415 conn(0x7f5f0c06fdb0 msgr2=0x7f5f0c0b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 shutdown_connections 2024-09-21T02:08:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:47.072+0000 7f5f11a36640 1 -- 172.21.15.96:0/1422354415 wait complete. 2024-09-21T02:08:47.097 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T02:08:47.097 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T02:08:47.098 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:47.098 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T02:08:47.109 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:47.109 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T02:08:47.161 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:47.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T02:08:47.213 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:08:47.213 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T02:08:47.266 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:47.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T02:08:47.280 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:47.280 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T02:08:47.333 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:47.334 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T02:08:47.389 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:08:47.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T02:08:47.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 3e39f309-6001-4136-ae9d-cef25153746c 0 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- 172.21.15.122:0/2132206275 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 msgr2=0x7f840c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 --2- 172.21.15.122:0/2132206275 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c06e840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8404009bf0 tx=0x7f840401cb20 comp rx=0 tx=0).stop 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- 172.21.15.122:0/2132206275 shutdown_connections 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 --2- 172.21.15.122:0/2132206275 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c06e840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- 172.21.15.122:0/2132206275 >> 172.21.15.122:0/2132206275 conn(0x7f840c0d1130 msgr2=0x7f840c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- 172.21.15.122:0/2132206275 shutdown_connections 2024-09-21T02:08:47.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- 172.21.15.122:0/2132206275 wait complete. 2024-09-21T02:08:47.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 Processor -- start 2024-09-21T02:08:47.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- start start 2024-09-21T02:08:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f840c15ba70 0x7f840c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f840c0b23c0 0x7f840c150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f840c16a340 con 0x7f840c15ba70 2024-09-21T02:08:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f840c16a1c0 con 0x7f840c0b23c0 2024-09-21T02:08:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.638+0000 7f84143df640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f840c16a4c0 con 0x7f840c152550 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841297c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f840c0b23c0 0x7f840c150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:59326/0 (socket says 172.21.15.122:59326) 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 -- 172.21.15.122:0/3928117202 learned_addr learned my addr 172.21.15.122:0/3928117202 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841197a640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f840c15ba70 0x7f840c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 -- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f840c0b23c0 msgr2=0x7f840c150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f840c0b23c0 0x7f840c150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 -- 172.21.15.122:0/3928117202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f840c15ba70 msgr2=0x7f840c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f840c15ba70 0x7f840c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 -- 172.21.15.122:0/3928117202 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c1fa930 con 0x7f840c152550 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841297c640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f840c0b23c0 0x7f840c150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f841217b640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c0ccc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f840401c590 tx=0x7f8404005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:47.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040b2040 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84040bb070 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c1fb540 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404004370 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c1faef0 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8404004510 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f84040c0c10 con 0x7f840c152550 2024-09-21T02:08:47.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.642+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840c06e840 con 0x7f840c152550 2024-09-21T02:08:47.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.646+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f84040167d0 con 0x7f840c152550 2024-09-21T02:08:47.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.718+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3e39f309-6001-4136-ae9d-cef25153746c", "id": 0} v 0) -- 0x7f840c15cd40 con 0x7f840c152550 2024-09-21T02:08:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f8402ffd640 1 -- 172.21.15.122:0/3928117202 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e39f309-6001-4136-ae9d-cef25153746c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f84040c0460 con 0x7f840c152550 2024-09-21T02:08:47.723 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-09-21T02:08:47.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 msgr2=0x7f840c0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c0ccc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f840401c590 tx=0x7f8404005a60 comp rx=0 tx=0).stop 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 shutdown_connections 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f840c0b23c0 0x7f840c150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f840c15ba70 0x7f840c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 --2- 172.21.15.122:0/3928117202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f840c152550 0x7f840c0ccc10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 >> 172.21.15.122:0/3928117202 conn(0x7f840c0d1130 msgr2=0x7f840c14e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 shutdown_connections 2024-09-21T02:08:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.722+0000 7f84143df640 1 -- 172.21.15.122:0/3928117202 wait complete. 2024-09-21T02:08:47.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 51c3338d-b835-47ea-8f9d-59786a4bd69e 1 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/3579698639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415be80 msgr2=0x7f5b94071750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/3579698639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415be80 0x7f5b94071750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b94073c30 tx=0x7f5b9001c800 comp rx=0 tx=0).stop 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/3579698639 shutdown_connections 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/3579698639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415be80 0x7f5b94071750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/3579698639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b9415ba70 0x7f5b94071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/3579698639 >> v2:172.21.15.96:3300/0 conn(0x7f5b94152550 0x7f5b9406e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/3579698639 >> 172.21.15.122:0/3579698639 conn(0x7f5b940d1130 msgr2=0x7f5b940d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/3579698639 shutdown_connections 2024-09-21T02:08:47.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/3579698639 wait complete. 2024-09-21T02:08:47.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 Processor -- start 2024-09-21T02:08:47.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- start start 2024-09-21T02:08:47.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 0x7f5b941fca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415ba70 0x7f5b9414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b9415be80 0x7f5b9406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9415f3f0 con 0x7f5b9415be80 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9415f270 con 0x7f5b9415ba70 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9b3b4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9415f570 con 0x7f5b94152550 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b99150640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 0x7f5b941fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9894f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415ba70 0x7f5b9414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b99150640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 0x7f5b941fca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:59352/0 (socket says 172.21.15.122:59352) 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b9894f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415ba70 0x7f5b9414a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32920/0 (socket says 172.21.15.122:32920) 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b99150640 1 -- 172.21.15.122:0/149959499 learned_addr learned my addr 172.21.15.122:0/149959499 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.894+0000 7f5b99951640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b9415be80 0x7f5b9406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99150640 1 -- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415ba70 msgr2=0x7f5b9414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99150640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415ba70 0x7f5b9414a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99150640 1 -- 172.21.15.122:0/149959499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b9415be80 msgr2=0x7f5b9406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99150640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b9415be80 0x7f5b9406ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99150640 1 -- 172.21.15.122:0/149959499 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9414ac40 con 0x7f5b94152550 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99951640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b9415be80 0x7f5b9406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b99150640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 0x7f5b941fca00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00ccb0 tx=0x7f5b8c00a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00f450 con 0x7f5b94152550 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b8c00f6e0 con 0x7f5b94152550 2024-09-21T02:08:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00faf0 con 0x7f5b94152550 2024-09-21T02:08:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b941543e0 con 0x7f5b94152550 2024-09-21T02:08:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941f9130 con 0x7f5b94152550 2024-09-21T02:08:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b8c00fc90 con 0x7f5b94152550 2024-09-21T02:08:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f5b8c018400 con 0x7f5b94152550 2024-09-21T02:08:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.898+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9406e840 con 0x7f5b94152550 2024-09-21T02:08:47.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.902+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5b8c014030 con 0x7f5b94152550 2024-09-21T02:08:47.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.974+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "51c3338d-b835-47ea-8f9d-59786a4bd69e", "id": 1} v 0) -- 0x7f5b94071ff0 con 0x7f5b94152550 2024-09-21T02:08:47.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b89ffb640 1 -- 172.21.15.122:0/149959499 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51c3338d-b835-47ea-8f9d-59786a4bd69e", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f5b8c005020 con 0x7f5b94152550 2024-09-21T02:08:47.979 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-21T02:08:47.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 msgr2=0x7f5b941fca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:47.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 0x7f5b941fca00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00ccb0 tx=0x7f5b8c00a620 comp rx=0 tx=0).stop 2024-09-21T02:08:47.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 shutdown_connections 2024-09-21T02:08:47.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b9415be80 0x7f5b9406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9415ba70 0x7f5b9414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 --2- 172.21.15.122:0/149959499 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b94152550 0x7f5b941fca00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:47.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 >> 172.21.15.122:0/149959499 conn(0x7f5b940d1130 msgr2=0x7f5b940d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:47.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 shutdown_connections 2024-09-21T02:08:47.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:47.978+0000 7f5b9b3b4640 1 -- 172.21.15.122:0/149959499 wait complete. 2024-09-21T02:08:47.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new ee46c0cd-1c7a-440c-aaa1-6888f5787b40 2 2024-09-21T02:08:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.158+0000 7fb7a211a640 1 -- 172.21.15.122:0/2930942951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0aee10 msgr2=0x7fb79c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.158+0000 7fb7a211a640 1 --2- 172.21.15.122:0/2930942951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0aee10 0x7fb79c06d830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb79c06d520 tx=0x7fb78c01c790 comp rx=0 tx=0).stop 2024-09-21T02:08:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.158+0000 7fb7a211a640 1 -- 172.21.15.122:0/2930942951 shutdown_connections 2024-09-21T02:08:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.158+0000 7fb7a211a640 1 --2- 172.21.15.122:0/2930942951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0aee10 0x7fb79c06d830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.162+0000 7fb7a211a640 1 --2- 172.21.15.122:0/2930942951 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 0x7fb79c0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.162+0000 7fb7a211a640 1 -- 172.21.15.122:0/2930942951 >> 172.21.15.122:0/2930942951 conn(0x7fb79c0ccaa0 msgr2=0x7fb79c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.162+0000 7fb7a211a640 1 -- 172.21.15.122:0/2930942951 shutdown_connections 2024-09-21T02:08:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.162+0000 7fb7a211a640 1 -- 172.21.15.122:0/2930942951 wait complete. 2024-09-21T02:08:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 Processor -- start 2024-09-21T02:08:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 -- start start 2024-09-21T02:08:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 0x7fb79c0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 0x7fb79c2065f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0cb780 0x7fb79c1611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c16ff70 con 0x7fb79c0cb780 2024-09-21T02:08:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1700f0 con 0x7fb79c0aee10 2024-09-21T02:08:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb7a211a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c170270 con 0x7fb79c0ae4c0 2024-09-21T02:08:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 0x7fb79c0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb79affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 0x7fb79c2065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb79affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 0x7fb79c2065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32922/0 (socket says 172.21.15.122:32922) 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb79affd640 1 -- 172.21.15.122:0/530733353 learned_addr learned my addr 172.21.15.122:0/530733353 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.166+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 0x7fb79c0b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:59356/0 (socket says 172.21.15.122:59356) 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79bfff640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0cb780 0x7fb79c1611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79affd640 1 -- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 msgr2=0x7fb79c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79affd640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 0x7fb79c0b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79affd640 1 -- 172.21.15.122:0/530733353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0cb780 msgr2=0x7fb79c1611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79affd640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0cb780 0x7fb79c1611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79affd640 1 -- 172.21.15.122:0/530733353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c0b4540 con 0x7fb79c0aee10 2024-09-21T02:08:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.170+0000 7fb79affd640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 0x7fb79c2065f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e970 tx=0x7fb79000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:48.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79000cd80 con 0x7fb79c0aee10 2024-09-21T02:08:48.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c0b4820 con 0x7fb79c0aee10 2024-09-21T02:08:48.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c0b4de0 con 0x7fb79c0aee10 2024-09-21T02:08:48.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb790004020 con 0x7fb79c0aee10 2024-09-21T02:08:48.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790004470 con 0x7fb79c0aee10 2024-09-21T02:08:48.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb790003990 con 0x7fb79c0aee10 2024-09-21T02:08:48.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb790021c10 con 0x7fb79c0aee10 2024-09-21T02:08:48.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.182+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c0052a0 con 0x7fb79c0aee10 2024-09-21T02:08:48.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.190+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb790038860 con 0x7fb79c0aee10 2024-09-21T02:08:48.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.278+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ee46c0cd-1c7a-440c-aaa1-6888f5787b40", "id": 2} v 0) -- 0x7fb77c005590 con 0x7fb79c0aee10 2024-09-21T02:08:48.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.282+0000 7fb783fff640 1 -- 172.21.15.122:0/530733353 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee46c0cd-1c7a-440c-aaa1-6888f5787b40", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb790026030 con 0x7fb79c0aee10 2024-09-21T02:08:48.285 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 msgr2=0x7fb79c2065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 0x7fb79c2065f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e970 tx=0x7fb79000ee30 comp rx=0 tx=0).stop 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 shutdown_connections 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb79c0cb780 0x7fb79c1611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb79c0aee10 0x7fb79c2065f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 --2- 172.21.15.122:0/530733353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb79c0ae4c0 0x7fb79c0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 >> 172.21.15.122:0/530733353 conn(0x7fb79c0ccaa0 msgr2=0x7fb79c162170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 shutdown_connections 2024-09-21T02:08:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.286+0000 7fb7a211a640 1 -- 172.21.15.122:0/530733353 wait complete. 2024-09-21T02:08:48.304 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 4e786efb-1547-443c-85e9-1ac57a64c176 3 2024-09-21T02:08:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 -- 172.21.15.122:0/3795149414 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 msgr2=0x7faa980cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 --2- 172.21.15.122:0/3795149414 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980cf010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009930 tx=0x7faa8c01c890 comp rx=0 tx=0).stop 2024-09-21T02:08:48.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 -- 172.21.15.122:0/3795149414 shutdown_connections 2024-09-21T02:08:48.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 --2- 172.21.15.122:0/3795149414 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980cf010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 -- 172.21.15.122:0/3795149414 >> 172.21.15.122:0/3795149414 conn(0x7faa9806eea0 msgr2=0x7faa9806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 -- 172.21.15.122:0/3795149414 shutdown_connections 2024-09-21T02:08:48.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.454+0000 7faa9da8a640 1 -- 172.21.15.122:0/3795149414 wait complete. 2024-09-21T02:08:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 Processor -- start 2024-09-21T02:08:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- start start 2024-09-21T02:08:48.457 INFO:tasks.ceph.mgr.y.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T02:08:48.457 INFO:tasks.ceph.mgr.y.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T02:08:48.458 INFO:tasks.ceph.mgr.y.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9806e2c0 0x7faa981fa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa980c4c90 0x7faa9814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa9816af90 con 0x7faa980c4c90 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa9816ae10 con 0x7faa9806e2c0 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faa9816b110 con 0x7faa980cec00 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9806e2c0 0x7faa981fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:59374/0 (socket says 172.21.15.122:59374) 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 -- 172.21.15.122:0/1452767710 learned_addr learned my addr 172.21.15.122:0/1452767710 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa96ffd640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa980c4c90 0x7faa9814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 -- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9806e2c0 msgr2=0x7faa981fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9806e2c0 0x7faa981fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 -- 172.21.15.122:0/1452767710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa980c4c90 msgr2=0x7faa9814ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa980c4c90 0x7faa9814ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 -- 172.21.15.122:0/1452767710 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981f6910 con 0x7faa980cec00 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa97fff640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980c3a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa9000c710 tx=0x7faa9000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90003b80 con 0x7faa980cec00 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa90003d20 con 0x7faa980cec00 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa98157020 con 0x7faa980cec00 2024-09-21T02:08:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9000aab0 con 0x7faa980cec00 2024-09-21T02:08:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981f6e70 con 0x7faa980cec00 2024-09-21T02:08:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa9000ac50 con 0x7faa980cec00 2024-09-21T02:08:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7faa900129a0 con 0x7faa980cec00 2024-09-21T02:08:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.458+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa980cf010 con 0x7faa980cec00 2024-09-21T02:08:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.462+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faa981f6910 con 0x7faa980cec00 2024-09-21T02:08:48.507 INFO:tasks.ceph.mgr.x.smithi122.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T02:08:48.507 INFO:tasks.ceph.mgr.x.smithi122.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T02:08:48.508 INFO:tasks.ceph.mgr.x.smithi122.stderr: from numpy import show_config as show_numpy_config 2024-09-21T02:08:48.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.530+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4e786efb-1547-443c-85e9-1ac57a64c176", "id": 3} v 0) -- 0x7faa98073310 con 0x7faa980cec00 2024-09-21T02:08:48.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.534+0000 7faa7ffff640 1 -- 172.21.15.122:0/1452767710 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4e786efb-1547-443c-85e9-1ac57a64c176", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7faa90012510 con 0x7faa980cec00 2024-09-21T02:08:48.536 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-09-21T02:08:48.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 msgr2=0x7faa980c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980c3a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa9000c710 tx=0x7faa9000cbd0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 shutdown_connections 2024-09-21T02:08:48.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa980cec00 0x7faa980c3a10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa980c4c90 0x7faa9814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 --2- 172.21.15.122:0/1452767710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9806e2c0 0x7faa981fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 >> 172.21.15.122:0/1452767710 conn(0x7faa9806eea0 msgr2=0x7faa980b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 shutdown_connections 2024-09-21T02:08:48.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.538+0000 7faa9da8a640 1 -- 172.21.15.122:0/1452767710 wait complete. 2024-09-21T02:08:48.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 09833021-edc8-4e94-ac1e-e0a54df57434 4 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 -- 172.21.15.122:0/1811964543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40d0710 msgr2=0x7f69d40d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 --2- 172.21.15.122:0/1811964543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40d0710 0x7f69d40d0b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69d4072030 tx=0x7f69cc01cb80 comp rx=0 tx=0).stop 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 -- 172.21.15.122:0/1811964543 shutdown_connections 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 --2- 172.21.15.122:0/1811964543 >> v2:172.21.15.96:3300/0 conn(0x7f69d4071920 0x7f69d40cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 --2- 172.21.15.122:0/1811964543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40d0710 0x7f69d40d0b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 -- 172.21.15.122:0/1811964543 >> 172.21.15.122:0/1811964543 conn(0x7f69d40d1130 msgr2=0x7f69d40d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 -- 172.21.15.122:0/1811964543 shutdown_connections 2024-09-21T02:08:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.706+0000 7f69db917640 1 -- 172.21.15.122:0/1811964543 wait complete. 2024-09-21T02:08:48.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 Processor -- start 2024-09-21T02:08:48.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- start start 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 0x7f69d4148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69d40d0710 0x7f69d41f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69d41490d0 0x7f69d41fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4169840 con 0x7f69d41490d0 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41696c0 con 0x7f69d40cd900 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f69d41699c0 con 0x7f69d40d0710 2024-09-21T02:08:48.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 0x7f69d4148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 0x7f69d4148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32952/0 (socket says 172.21.15.122:32952) 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 -- 172.21.15.122:0/64969720 learned_addr learned my addr 172.21.15.122:0/64969720 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d8eb2640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69d40d0710 0x7f69d41f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d9eb4640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69d41490d0 0x7f69d41fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 -- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69d40d0710 msgr2=0x7f69d41f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69d40d0710 0x7f69d41f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 -- 172.21.15.122:0/64969720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69d41490d0 msgr2=0x7f69d41fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69d41490d0 0x7f69d41fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 -- 172.21.15.122:0/64969720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d415b640 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69d96b3640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 0x7f69d4148b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009b10 tx=0x7f69cc0a5940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0bb070 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69cc004e00 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d415c0b0 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0b2040 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d415bb90 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f69cc0be070 con 0x7f69d40cd900 2024-09-21T02:08:48.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f69cc005770 con 0x7f69d40cd900 2024-09-21T02:08:48.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.710+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d40cdd10 con 0x7f69d40cd900 2024-09-21T02:08:48.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.714+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f69cc0c5070 con 0x7f69d40cd900 2024-09-21T02:08:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.782+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "09833021-edc8-4e94-ac1e-e0a54df57434", "id": 4} v 0) -- 0x7f69d414a150 con 0x7f69d40cd900 2024-09-21T02:08:48.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.790+0000 7f69c9ffb640 1 -- 172.21.15.122:0/64969720 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09833021-edc8-4e94-ac1e-e0a54df57434", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f69cc016770 con 0x7f69d40cd900 2024-09-21T02:08:48.792 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-21T02:08:48.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 msgr2=0x7f69d4148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 0x7f69d4148b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009b10 tx=0x7f69cc0a5940 comp rx=0 tx=0).stop 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 shutdown_connections 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69d41490d0 0x7f69d41fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69d40d0710 0x7f69d41f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 --2- 172.21.15.122:0/64969720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d40cd900 0x7f69d4148b30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 >> 172.21.15.122:0/64969720 conn(0x7f69d40d1130 msgr2=0x7f69d415dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 shutdown_connections 2024-09-21T02:08:48.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.794+0000 7f69db917640 1 -- 172.21.15.122:0/64969720 wait complete. 2024-09-21T02:08:48.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 562a7d36-d24f-48ea-9dd8-cb8366e22cb7 5 2024-09-21T02:08:48.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/3371615690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6415ede0 msgr2=0x7f4e6415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/3371615690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6415ede0 0x7f4e6415f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e64069380 tx=0x7f4e5c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/3371615690 shutdown_connections 2024-09-21T02:08:48.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/3371615690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6415ede0 0x7f4e6415f1f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/3371615690 >> v2:172.21.15.122:3300/0 conn(0x7f4e6406fd90 0x7f4e640701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/3371615690 >> 172.21.15.122:0/3371615690 conn(0x7f4e640d0740 msgr2=0x7f4e640d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:48.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/3371615690 shutdown_connections 2024-09-21T02:08:48.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.954+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/3371615690 wait complete. 2024-09-21T02:08:48.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 Processor -- start 2024-09-21T02:08:48.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 -- start start 2024-09-21T02:08:48.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 0x7f4e6414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e6415b250 0x7f4e6406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415ede0 0x7f4e6406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64168c70 con 0x7f4e6415b250 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64168af0 con 0x7f4e6415ede0 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e6b0ab640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4e64168df0 con 0x7f4e6406fd90 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 0x7f4e6414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 0x7f4e6414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:59422/0 (socket says 172.21.15.122:59422) 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 -- 172.21.15.122:0/915256333 learned_addr learned my addr 172.21.15.122:0/915256333 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e69648640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415ede0 0x7f4e6406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e63fff640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e6415b250 0x7f4e6406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:48.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 -- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415ede0 msgr2=0x7f4e6406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415ede0 0x7f4e6406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 -- 172.21.15.122:0/915256333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e6415b250 msgr2=0x7f4e6406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e6415b250 0x7f4e6406ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 -- 172.21.15.122:0/915256333 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e641f6ac0 con 0x7f4e6406fd90 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e68e47640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 0x7f4e6414da30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c3f0 tx=0x7f4e5000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.958+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50017070 con 0x7f4e6406fd90 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e500053c0 con 0x7f4e6406fd90 2024-09-21T02:08:48.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50003860 con 0x7f4e6406fd90 2024-09-21T02:08:48.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e64157c10 con 0x7f4e6406fd90 2024-09-21T02:08:48.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e64158190 con 0x7f4e6406fd90 2024-09-21T02:08:48.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e5001a070 con 0x7f4e6406fd90 2024-09-21T02:08:48.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4e50011bd0 con 0x7f4e6406fd90 2024-09-21T02:08:48.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e640701a0 con 0x7f4e6406fd90 2024-09-21T02:08:48.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:48.962+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4e50021080 con 0x7f4e6406fd90 2024-09-21T02:08:49.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.034+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "562a7d36-d24f-48ea-9dd8-cb8366e22cb7", "id": 5} v 0) -- 0x7f4e640b53c0 con 0x7f4e6406fd90 2024-09-21T02:08:49.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.038+0000 7f4e617fa640 1 -- 172.21.15.122:0/915256333 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "562a7d36-d24f-48ea-9dd8-cb8366e22cb7", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4e50011590 con 0x7f4e6406fd90 2024-09-21T02:08:49.040 INFO:teuthology.orchestra.run.smithi122.stdout:5 2024-09-21T02:08:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 msgr2=0x7f4e6414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 0x7f4e6414da30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c3f0 tx=0x7f4e5000c8b0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 shutdown_connections 2024-09-21T02:08:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415ede0 0x7f4e6406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e6415b250 0x7f4e6406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 --2- 172.21.15.122:0/915256333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e6406fd90 0x7f4e6414da30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 >> 172.21.15.122:0/915256333 conn(0x7f4e640d0740 msgr2=0x7f4e64151240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 shutdown_connections 2024-09-21T02:08:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.042+0000 7f4e6b0ab640 1 -- 172.21.15.122:0/915256333 wait complete. 2024-09-21T02:08:49.058 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 984413c0-d6d2-4d1f-8675-a3ad89f2939c 6 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 -- 172.21.15.122:0/2948805944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 msgr2=0x7fd8680b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2948805944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680b45b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8680b1da0 tx=0x7fd85801c840 comp rx=0 tx=0).stop 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 -- 172.21.15.122:0/2948805944 shutdown_connections 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2948805944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680b45b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2948805944 >> v2:172.21.15.96:3300/0 conn(0x7fd8681601a0 0x7fd8681605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2948805944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 0x7fd8680cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 -- 172.21.15.122:0/2948805944 >> 172.21.15.122:0/2948805944 conn(0x7fd8680cf150 msgr2=0x7fd8680cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 -- 172.21.15.122:0/2948805944 shutdown_connections 2024-09-21T02:08:49.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.198+0000 7fd86d23b640 1 -- 172.21.15.122:0/2948805944 wait complete. 2024-09-21T02:08:49.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 Processor -- start 2024-09-21T02:08:49.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- start start 2024-09-21T02:08:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 0x7fd868159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8681601a0 0x7fd868159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd868168cc0 con 0x7fd8681601a0 2024-09-21T02:08:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd868168b40 con 0x7fd8680b41a0 2024-09-21T02:08:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd868168e40 con 0x7fd868156bd0 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86659c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 0x7fd868159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86659c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 0x7fd868159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:59440/0 (socket says 172.21.15.122:59440) 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86659c640 1 -- 172.21.15.122:0/2303539740 learned_addr learned my addr 172.21.15.122:0/2303539740 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86759e640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8681601a0 0x7fd868159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 -- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 msgr2=0x7fd868159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 0x7fd868159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 -- 172.21.15.122:0/2303539740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8681601a0 msgr2=0x7fd868159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8681601a0 0x7fd868159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 -- 172.21.15.122:0/2303539740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86815fdd0 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86759e640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8681601a0 0x7fd868159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd866d9d640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680c5120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9c0 tx=0x7fd85000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85000ccc0 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd850004020 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8680ca0e0 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850004450 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8680b62d0 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd850004b40 con 0x7fd8680b41a0 2024-09-21T02:08:49.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fd850003c90 con 0x7fd8680b41a0 2024-09-21T02:08:49.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.202+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd868159e80 con 0x7fd8680b41a0 2024-09-21T02:08:49.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.206+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd868159e80 con 0x7fd8680b41a0 2024-09-21T02:08:49.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.270+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "984413c0-d6d2-4d1f-8675-a3ad89f2939c", "id": 6} v 0) -- 0x7fd86806a170 con 0x7fd8680b41a0 2024-09-21T02:08:49.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd85f7fe640 1 -- 172.21.15.122:0/2303539740 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "984413c0-d6d2-4d1f-8675-a3ad89f2939c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fd8500036a0 con 0x7fd8680b41a0 2024-09-21T02:08:49.279 INFO:teuthology.orchestra.run.smithi122.stdout:6 2024-09-21T02:08:49.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 msgr2=0x7fd8680c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680c5120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9c0 tx=0x7fd85000ee80 comp rx=0 tx=0).stop 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 shutdown_connections 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8681601a0 0x7fd868159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd868156bd0 0x7fd868159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 --2- 172.21.15.122:0/2303539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8680b41a0 0x7fd8680c5120 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 >> 172.21.15.122:0/2303539740 conn(0x7fd8680cf150 msgr2=0x7fd8681544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 shutdown_connections 2024-09-21T02:08:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.278+0000 7fd86d23b640 1 -- 172.21.15.122:0/2303539740 wait complete. 2024-09-21T02:08:49.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new fcd138eb-65fc-4ab2-b493-3935f86d7c8c 7 2024-09-21T02:08:49.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- 172.21.15.122:0/2395217255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8415ba70 msgr2=0x7f9b84071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- 172.21.15.122:0/2395217255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8415ba70 0x7f9b84071210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b840723a0 tx=0x7f9b7001c800 comp rx=0 tx=0).stop 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- 172.21.15.122:0/2395217255 shutdown_connections 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- 172.21.15.122:0/2395217255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8415ba70 0x7f9b84071210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- 172.21.15.122:0/2395217255 >> v2:172.21.15.96:3300/0 conn(0x7f9b8415be80 0x7f9b84071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- 172.21.15.122:0/2395217255 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b84152550 0x7f9b8406e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- 172.21.15.122:0/2395217255 >> 172.21.15.122:0/2395217255 conn(0x7f9b840d1130 msgr2=0x7f9b840d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- 172.21.15.122:0/2395217255 shutdown_connections 2024-09-21T02:08:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- 172.21.15.122:0/2395217255 wait complete. 2024-09-21T02:08:49.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 Processor -- start 2024-09-21T02:08:49.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- start start 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 0x7f9b84156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8415ba70 0x7f9b84150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b8415be80 0x7f9b841514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8415f3f0 con 0x7f9b8415ba70 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8415f270 con 0x7f9b84152550 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b8b264640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8415f570 con 0x7f9b8415be80 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 0x7f9b84156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89801640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b8415be80 0x7f9b841514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 0x7f9b84156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33016/0 (socket says 172.21.15.122:33016) 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 -- 172.21.15.122:0/386906583 learned_addr learned my addr 172.21.15.122:0/386906583 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-21T02:08:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b83fff640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8415ba70 0x7f9b84150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 -- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b8415be80 msgr2=0x7f9b841514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b8415be80 0x7f9b841514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 -- 172.21.15.122:0/386906583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8415ba70 msgr2=0x7f9b84150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8415ba70 0x7f9b84150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 -- 172.21.15.122:0/386906583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b84158de0 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89801640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b8415be80 0x7f9b841514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.438+0000 7f9b89000640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 0x7f9b84156750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800e9f0 tx=0x7f9b7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800cd80 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b78004020 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8414d370 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78004470 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8414de50 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b78004b60 con 0x7f9b84152550 2024-09-21T02:08:49.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f9b780037e0 con 0x7f9b84152550 2024-09-21T02:08:49.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8414da80 con 0x7f9b84152550 2024-09-21T02:08:49.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.442+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9b78003b00 con 0x7f9b84152550 2024-09-21T02:08:49.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.514+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fcd138eb-65fc-4ab2-b493-3935f86d7c8c", "id": 7} v 0) -- 0x7f9b8406e840 con 0x7f9b84152550 2024-09-21T02:08:49.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.518+0000 7f9b817fa640 1 -- 172.21.15.122:0/386906583 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fcd138eb-65fc-4ab2-b493-3935f86d7c8c", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9b78003ce0 con 0x7f9b84152550 2024-09-21T02:08:49.521 INFO:teuthology.orchestra.run.smithi122.stdout:7 2024-09-21T02:08:49.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 msgr2=0x7f9b84156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 0x7f9b84156750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800e9f0 tx=0x7f9b7800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 shutdown_connections 2024-09-21T02:08:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b8415be80 0x7f9b841514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8415ba70 0x7f9b84150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 --2- 172.21.15.122:0/386906583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84152550 0x7f9b84156750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 >> 172.21.15.122:0/386906583 conn(0x7f9b840d1130 msgr2=0x7f9b8415a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 shutdown_connections 2024-09-21T02:08:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:08:49.522+0000 7f9b8b264640 1 -- 172.21.15.122:0/386906583 wait complete. 2024-09-21T02:08:49.539 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T02:08:49.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T02:08:49.541 INFO:tasks.ceph.osd.0:Started 2024-09-21T02:08:49.541 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T02:08:49.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T02:08:49.544 INFO:tasks.ceph.osd.1:Started 2024-09-21T02:08:49.544 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T02:08:49.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T02:08:49.546 INFO:tasks.ceph.osd.2:Started 2024-09-21T02:08:49.546 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T02:08:49.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T02:08:49.548 INFO:tasks.ceph.osd.3:Started 2024-09-21T02:08:49.548 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T02:08:49.548 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T02:08:49.550 INFO:tasks.ceph.osd.4:Started 2024-09-21T02:08:49.551 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T02:08:49.551 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T02:08:49.552 INFO:tasks.ceph.osd.5:Started 2024-09-21T02:08:49.553 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T02:08:49.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T02:08:49.554 INFO:tasks.ceph.osd.6:Started 2024-09-21T02:08:49.554 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T02:08:49.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T02:08:49.559 INFO:tasks.ceph.osd.7:Started 2024-09-21T02:08:49.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:08:49.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- 172.21.15.96:0/2211082195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180071920 msgr2=0x7f71800cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 --2- 172.21.15.96:0/2211082195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180071920 0x7f71800cdd10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7178009bc0 tx=0x7f717801cba0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- 172.21.15.96:0/2211082195 shutdown_connections 2024-09-21T02:08:49.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 --2- 172.21.15.96:0/2211082195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180071920 0x7f71800cdd10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 --2- 172.21.15.96:0/2211082195 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71800d0710 0x7f71800d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- 172.21.15.96:0/2211082195 >> 172.21.15.96:0/2211082195 conn(0x7f71800d1130 msgr2=0x7f71800d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- 172.21.15.96:0/2211082195 shutdown_connections 2024-09-21T02:08:49.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- 172.21.15.96:0/2211082195 wait complete. 2024-09-21T02:08:49.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 Processor -- start 2024-09-21T02:08:49.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- start start 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 0x7f718015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71800d0710 0x7f7180146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71801f7190 0x7f71801466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7180169a00 con 0x7f71800cd900 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7180169880 con 0x7f71800d0710 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f7187300640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7180169b80 con 0x7f71801f7190 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f718509c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 0x7f718015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f718509c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 0x7f718015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58122/0 (socket says 172.21.15.96:58122) 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f718509c640 1 -- 172.21.15.96:0/88084936 learned_addr learned my addr 172.21.15.96:0/88084936 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f718589d640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71801f7190 0x7f71801466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.716+0000 7f718489b640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71800d0710 0x7f7180146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f718509c640 1 -- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71801f7190 msgr2=0x7f71801466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f718509c640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71801f7190 0x7f71801466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f718509c640 1 -- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71800d0710 msgr2=0x7f7180146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f718509c640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71800d0710 0x7f7180146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f718509c640 1 -- 172.21.15.96:0/88084936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718015b180 con 0x7f71800cd900 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f718509c640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 0x7f718015a5c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f717400e7d0 tx=0x7f717400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400ccc0 con 0x7f71800cd900 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7174010430 con 0x7f71800cd900 2024-09-21T02:08:49.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174010860 con 0x7f71800cd900 2024-09-21T02:08:49.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718015b460 con 0x7f71800cd900 2024-09-21T02:08:49.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801476a0 con 0x7f71800cd900 2024-09-21T02:08:49.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7174010a00 con 0x7f71800cd900 2024-09-21T02:08:49.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7174018c40 con 0x7f71800cd900 2024-09-21T02:08:49.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.720+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71800cdd10 con 0x7f71800cd900 2024-09-21T02:08:49.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.724+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7174010cf0 con 0x7f71800cd900 2024-09-21T02:08:49.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.784+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7180150af0 con 0x7f71800cd900 2024-09-21T02:08:49.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.784+0000 7f7171ffb640 1 -- 172.21.15.96:0/88084936 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7174018420 con 0x7f71800cd900 2024-09-21T02:08:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:08:49.791 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:08:49.514910+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:08:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 msgr2=0x7f718015a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 0x7f718015a5c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f717400e7d0 tx=0x7f717400ec90 comp rx=0 tx=0).stop 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 shutdown_connections 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71801f7190 0x7f71801466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71800d0710 0x7f7180146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 --2- 172.21.15.96:0/88084936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71800cd900 0x7f718015a5c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 >> 172.21.15.96:0/88084936 conn(0x7f71800d1130 msgr2=0x7f718015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 shutdown_connections 2024-09-21T02:08:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:08:49.788+0000 7f7187300640 1 -- 172.21.15.96:0/88084936 wait complete. 2024-09-21T02:08:49.809 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T02:08:49.810 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T02:08:50.616 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:08:50.764 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 -- 172.21.15.96:0/478186323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e815c3e0 msgr2=0x7f71e80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.764 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/478186323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e815c3e0 0x7f71e80cbaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71e80757f0 tx=0x7f71e001cd80 comp rx=0 tx=0).stop 2024-09-21T02:08:50.764 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 -- 172.21.15.96:0/478186323 shutdown_connections 2024-09-21T02:08:50.764 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/478186323 >> v2:172.21.15.122:3300/0 conn(0x7f71e806e3e0 0x7f71e815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.764 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/478186323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e815c3e0 0x7f71e80cbaf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.765 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/478186323 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71e80ced00 0x7f71e806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.765 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 -- 172.21.15.96:0/478186323 >> 172.21.15.96:0/478186323 conn(0x7f71e806eea0 msgr2=0x7f71e806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:50.765 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 -- 172.21.15.96:0/478186323 shutdown_connections 2024-09-21T02:08:50.765 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 -- 172.21.15.96:0/478186323 wait complete. 2024-09-21T02:08:50.765 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 Processor -- start 2024-09-21T02:08:50.765 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.760+0000 7f71ed8b0640 1 -- start start 2024-09-21T02:08:50.768 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 0x7f71e8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.768 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71e80ced00 0x7f71e8159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.768 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71e815c3e0 0x7f71e80b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.768 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71e80caab0 con 0x7f71e806e3e0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71e80ca930 con 0x7f71e815c3e0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f71e80cac30 con 0x7f71e80ced00 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 0x7f71e8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 0x7f71e8154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58146/0 (socket says 172.21.15.96:58146) 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 -- 172.21.15.96:0/4040805346 learned_addr learned my addr 172.21.15.96:0/4040805346 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 -- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71e80ced00 msgr2=0x7f71e8159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e77fe640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71e815c3e0 0x7f71e80b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71de7fc640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71e80ced00 0x7f71e8159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71e80ced00 0x7f71e8159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 -- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71e815c3e0 msgr2=0x7f71e80b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71e815c3e0 0x7f71e80b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 -- 172.21.15.96:0/4040805346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e80c9700 con 0x7f71e806e3e0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e6ffd640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 0x7f71e8154460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c940 tx=0x7f71d800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:50.769 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004220 con 0x7f71e806e3e0 2024-09-21T02:08:50.770 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71d80079b0 con 0x7f71e806e3e0 2024-09-21T02:08:50.770 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e80c9c00 con 0x7f71e806e3e0 2024-09-21T02:08:50.770 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80044b0 con 0x7f71e806e3e0 2024-09-21T02:08:50.770 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e80b4ed0 con 0x7f71e806e3e0 2024-09-21T02:08:50.770 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e80cb170 con 0x7f71e806e3e0 2024-09-21T02:08:50.771 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 2) ==== 48655+0+0 (secure 0 0 0) 0x7f71d8003600 con 0x7f71e806e3e0 2024-09-21T02:08:50.771 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.764+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f71d804eeb0 con 0x7f71e806e3e0 2024-09-21T02:08:50.771 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.768+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f71d8014030 con 0x7f71e806e3e0 2024-09-21T02:08:50.831 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71e80af920 con 0x7f71e806e3e0 2024-09-21T02:08:50.832 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71e4ff9640 1 -- 172.21.15.96:0/4040805346 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f71d801fd00 con 0x7f71e806e3e0 2024-09-21T02:08:50.832 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-21T02:08:50.832 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":10,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:08:49.514910+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:08:50.833 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 msgr2=0x7f71e8154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.833 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 0x7f71e8154460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c940 tx=0x7f71d800ce00 comp rx=0 tx=0).stop 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 shutdown_connections 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71e815c3e0 0x7f71e80b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71e80ced00 0x7f71e8159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 --2- 172.21.15.96:0/4040805346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e806e3e0 0x7f71e8154460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 >> 172.21.15.96:0/4040805346 conn(0x7f71e806eea0 msgr2=0x7f71e80b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 shutdown_connections 2024-09-21T02:08:50.834 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:50.828+0000 7f71ed8b0640 1 -- 172.21.15.96:0/4040805346 wait complete. 2024-09-21T02:08:50.850 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T02:08:50.899 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T02:08:50.898+0000 7fe0b1bca7c0 -1 Falling back to public interface 2024-09-21T02:08:50.953 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-21T02:08:50.948+0000 7f107a0897c0 -1 Falling back to public interface 2024-09-21T02:08:51.009 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-21T02:08:51.004+0000 7fb0652da7c0 -1 Falling back to public interface 2024-09-21T02:08:51.149 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-21T02:08:51.144+0000 7fa1894c67c0 -1 Falling back to public interface 2024-09-21T02:08:51.199 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T02:08:51.198+0000 7fe25784a7c0 -1 Falling back to public interface 2024-09-21T02:08:51.216 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-21T02:08:51.212+0000 7f1ea73f67c0 -1 Falling back to public interface 2024-09-21T02:08:51.258 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-21T02:08:51.258+0000 7f2b32e287c0 -1 Falling back to public interface 2024-09-21T02:08:51.259 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-21T02:08:51.258+0000 7eff7f6617c0 -1 Falling back to public interface 2024-09-21T02:08:51.507 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T02:08:51.506+0000 7f2bbcc50640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T02:08:51.507 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T02:08:51.506+0000 7f2bbcc50640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T02:08:54.949 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-21T02:08:54.944+0000 7f107a0897c0 -1 osd.0 0 log_to_monitors true 2024-09-21T02:08:54.988 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T02:08:54.986+0000 7fe0b1bca7c0 -1 osd.4 0 log_to_monitors true 2024-09-21T02:08:55.065 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-21T02:08:55.060+0000 7fb0652da7c0 -1 osd.3 0 log_to_monitors true 2024-09-21T02:08:55.151 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T02:08:55.150+0000 7fe25784a7c0 -1 osd.5 0 log_to_monitors true 2024-09-21T02:08:55.229 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-21T02:08:55.224+0000 7f1ea73f67c0 -1 osd.2 0 log_to_monitors true 2024-09-21T02:08:55.595 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-21T02:08:55.594+0000 7eff7f6617c0 -1 osd.6 0 log_to_monitors true 2024-09-21T02:08:55.611 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-21T02:08:55.610+0000 7f2b32e287c0 -1 osd.7 0 log_to_monitors true 2024-09-21T02:08:55.671 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-21T02:08:55.668+0000 7fa1894c67c0 -1 osd.1 0 log_to_monitors true 2024-09-21T02:08:57.657 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:08:57.837 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 -- 172.21.15.96:0/1116050954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 msgr2=0x7f45f40b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.837 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 --2- 172.21.15.96:0/1116050954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 0x7f45f40b45b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009840 tx=0x7f45dc01c820 comp rx=0 tx=0).stop 2024-09-21T02:08:57.837 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 -- 172.21.15.96:0/1116050954 shutdown_connections 2024-09-21T02:08:57.837 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 --2- 172.21.15.96:0/1116050954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 0x7f45f40b45b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.837 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 --2- 172.21.15.96:0/1116050954 >> v2:172.21.15.122:3300/0 conn(0x7f45f4156bd0 0x7f45f40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.838 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 -- 172.21.15.96:0/1116050954 >> 172.21.15.96:0/1116050954 conn(0x7f45f40cf150 msgr2=0x7f45f40cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.838 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 -- 172.21.15.96:0/1116050954 shutdown_connections 2024-09-21T02:08:57.838 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.832+0000 7f45f9222640 1 -- 172.21.15.96:0/1116050954 wait complete. 2024-09-21T02:08:57.843 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.836+0000 7f45f9222640 1 Processor -- start 2024-09-21T02:08:57.843 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.836+0000 7f45f9222640 1 -- start start 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f9222640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 0x7f45f40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f9222640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f4156bd0 0x7f45f40c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f9222640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 0x7f45f40c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f9222640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4169ee0 con 0x7f45f41601a0 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f9222640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4169d60 con 0x7f45f4156bd0 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f9222640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f45f416a060 con 0x7f45f40b41a0 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 0x7f45f40c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 0x7f45f40c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50978/0 (socket says 172.21.15.96:50978) 2024-09-21T02:08:57.844 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 -- 172.21.15.96:0/2468773566 learned_addr learned my addr 172.21.15.96:0/2468773566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:08:57.845 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f2d9d640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 0x7f45f40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.845 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 -- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 msgr2=0x7f45f40cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.845 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 0x7f45f40cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.845 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 -- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f4156bd0 msgr2=0x7f45f40c5e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f4156bd0 0x7f45f40c5e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.840+0000 7f45f359e640 1 -- 172.21.15.96:0/2468773566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f41563e0 con 0x7f45f41601a0 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45f359e640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 0x7f45f40c9e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c760 tx=0x7f45ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003b80 con 0x7f45f41601a0 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ec003d40 con 0x7f45f41601a0 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00aab0 con 0x7f45f41601a0 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f40b2f70 con 0x7f45f41601a0 2024-09-21T02:08:57.847 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f40c8560 con 0x7f45f41601a0 2024-09-21T02:08:57.848 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f415f6f0 con 0x7f45f41601a0 2024-09-21T02:08:57.849 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f45ec00ac50 con 0x7f45f41601a0 2024-09-21T02:08:57.850 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.844+0000 7f45ebfff640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f45d40731a0 0x7f45d40756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.850 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.848+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f45ec093180 con 0x7f45f41601a0 2024-09-21T02:08:57.850 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.848+0000 7f45f2d9d640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f45d40731a0 0x7f45d40756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.851 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.848+0000 7f45f2d9d640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f45d40731a0 0x7f45d40756b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009b40 tx=0x7f45e4009260 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.857 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:57.852+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f45f40b4af0 con 0x7f45f41601a0 2024-09-21T02:08:58.051 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T02:08:58.050+0000 7fe0adb70640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T02:08:58.090 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T02:08:58.090+0000 7fe0a596d640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.099 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T02:08:58.098+0000 7fe254005640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T02:08:58.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.116+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45f40b45b0 con 0x7f45f41601a0 2024-09-21T02:08:58.120 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.116+0000 7f45ebfff640 1 -- 172.21.15.96:0/2468773566 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f45ec05f000 con 0x7f45f41601a0 2024-09-21T02:08:58.120 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-21T02:08:58.120 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":13,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:08:57.515052+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:08:58.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f45d40731a0 msgr2=0x7f45d40756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f45d40731a0 0x7f45d40756b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009b40 tx=0x7f45e4009260 comp rx=0 tx=0).stop 2024-09-21T02:08:58.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 msgr2=0x7f45f40c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 0x7f45f40c9e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c760 tx=0x7f45ec00cc20 comp rx=0 tx=0).stop 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 shutdown_connections 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f45d40731a0 0x7f45d40756b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45f41601a0 0x7f45f40c9e40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f4156bd0 0x7f45f40c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 --2- 172.21.15.96:0/2468773566 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f45f40b41a0 0x7f45f40cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 >> 172.21.15.96:0/2468773566 conn(0x7f45f40cf150 msgr2=0x7f45f4155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 shutdown_connections 2024-09-21T02:08:58.126 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:08:58.120+0000 7f45f9222640 1 -- 172.21.15.96:0/2468773566 wait complete. 2024-09-21T02:08:58.144 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T02:08:58.142+0000 7fe24b5ed640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.148 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T02:08:58.461 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-21T02:08:58.456+0000 7f1ea3bb1640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T02:08:58.492 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-21T02:08:58.488+0000 7f1e9b199640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.622 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-21T02:08:58.616+0000 7f1076844640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T02:08:58.623 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-21T02:08:58.620+0000 7fb061a95640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T02:08:58.653 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-21T02:08:58.648+0000 7f106de2c640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.653 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-21T02:08:58.648+0000 7fb05907d640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.830 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-21T02:08:58.824+0000 7fa185c81640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T02:08:58.868 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-21T02:08:58.864+0000 7fa17d269640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.927 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-21T02:08:58.926+0000 7f2b2edce640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T02:08:58.950 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-21T02:08:58.950+0000 7f2b26bcb640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:08:58.964 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-21T02:08:58.962+0000 7eff7b607640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T02:08:58.982 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-21T02:08:58.982+0000 7eff73404640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:09:01.951 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:09:01.954 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:09:04.959 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:05.118 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 -- 172.21.15.96:0/950966406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c0cfac0 msgr2=0x7f5e6c15ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.118 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/950966406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c15ef70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009900 tx=0x7f5e5c01c5e0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.118 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 -- 172.21.15.96:0/950966406 shutdown_connections 2024-09-21T02:09:05.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/950966406 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e6c074310 0x7f5e6c0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/950966406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c15ef70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 -- 172.21.15.96:0/950966406 >> 172.21.15.96:0/950966406 conn(0x7f5e6c0d13d0 msgr2=0x7f5e6c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:05.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 -- 172.21.15.96:0/950966406 shutdown_connections 2024-09-21T02:09:05.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.112+0000 7f5e72ee7640 1 -- 172.21.15.96:0/950966406 wait complete. 2024-09-21T02:09:05.119 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 Processor -- start 2024-09-21T02:09:05.120 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 -- start start 2024-09-21T02:09:05.122 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.122 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e6c0cfed0 0x7f5e6c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.122 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 0x7f5e6c154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.122 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c06ff00 con 0x7f5e6c154260 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c06fd80 con 0x7f5e6c0cfac0 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6c070080 con 0x7f5e6c0cfed0 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 0x7f5e6c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 0x7f5e6c154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51002/0 (socket says 172.21.15.96:51002) 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 -- 172.21.15.96:0/2819743792 learned_addr learned my addr 172.21.15.96:0/2819743792 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e70c83640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e6bfff640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e6c0cfed0 0x7f5e6c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 -- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e6c0cfed0 msgr2=0x7f5e6c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e6c0cfed0 0x7f5e6c1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 -- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e6c0cfac0 msgr2=0x7f5e6c14c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c14c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.123 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 -- 172.21.15.96:0/2819743792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c154bb0 con 0x7f5e6c154260 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e70c83640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e71484640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 0x7f5e6c154670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400e940 tx=0x7f5e6400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64004d30 con 0x7f5e6c154260 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e6400cb20 con 0x7f5e6c154260 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64005020 con 0x7f5e6c154260 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6c14f7a0 con 0x7f5e6c154260 2024-09-21T02:09:05.124 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.116+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6c14d930 con 0x7f5e6c154260 2024-09-21T02:09:05.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.120+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6c0af4c0 con 0x7f5e6c154260 2024-09-21T02:09:05.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.120+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f5e640051c0 con 0x7f5e6c154260 2024-09-21T02:09:05.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.120+0000 7f5e697fa640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5e58073170 0x7f5e58075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.120+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5e64094880 con 0x7f5e6c154260 2024-09-21T02:09:05.125 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.120+0000 7f5e70c83640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5e58073170 0x7f5e58075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.126 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.120+0000 7f5e70c83640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5e58073170 0x7f5e58075680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0af000 tx=0x7f5e5c01ae20 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:05.129 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.124+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5e6c154bb0 con 0x7f5e6c154260 2024-09-21T02:09:05.164 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.160+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e6405b9b0 con 0x7f5e6c154260 2024-09-21T02:09:05.306 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.300+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e6c14d6e0 con 0x7f5e6c154260 2024-09-21T02:09:05.307 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e697fa640 1 -- 172.21.15.96:0/2819743792 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12896 (secure 0 0 0) 0x7f5e64016030 con 0x7f5e6c154260 2024-09-21T02:09:05.307 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-21T02:09:05.308 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":18,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:03.132698+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5e58073170 msgr2=0x7f5e58075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5e58073170 0x7f5e58075680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0af000 tx=0x7f5e5c01ae20 comp rx=0 tx=0).stop 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 msgr2=0x7f5e6c154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 0x7f5e6c154670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400e940 tx=0x7f5e6400ee00 comp rx=0 tx=0).stop 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 shutdown_connections 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5e58073170 0x7f5e58075680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e6c154260 0x7f5e6c154670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.310 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e6c0cfed0 0x7f5e6c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.311 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 --2- 172.21.15.96:0/2819743792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e6c0cfac0 0x7f5e6c14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.311 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 >> 172.21.15.96:0/2819743792 conn(0x7f5e6c0d13d0 msgr2=0x7f5e6c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:05.311 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 shutdown_connections 2024-09-21T02:09:05.311 INFO:teuthology.misc.health.smithi096.stderr:2024-09-21T02:09:05.304+0000 7f5e72ee7640 1 -- 172.21.15.96:0/2819743792 wait complete. 2024-09-21T02:09:05.328 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T02:09:05.328 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T02:09:05.328 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:09:05.328 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:09:05.330 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:09:05.330 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:09:05.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:09:05.333 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:09:05.333 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:09:05.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:09:05.335 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:09:05.336 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:09:05.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:09:05.338 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:09:05.338 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T02:09:05.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:05.440 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-21T02:09:05.440 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2024-09-21T02:09:05.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 -- 172.21.15.96:0/2249840611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c0b41a0 msgr2=0x7f149c0b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/2249840611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c0b41a0 0x7f149c0b45b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f148c009bc0 tx=0x7f148c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 -- 172.21.15.96:0/2249840611 shutdown_connections 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/2249840611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c0b41a0 0x7f149c0b45b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/2249840611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f149c156bd0 0x7f149c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 -- 172.21.15.96:0/2249840611 >> 172.21.15.96:0/2249840611 conn(0x7f149c0cf150 msgr2=0x7f149c0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 -- 172.21.15.96:0/2249840611 shutdown_connections 2024-09-21T02:09:05.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 -- 172.21.15.96:0/2249840611 wait complete. 2024-09-21T02:09:05.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 Processor -- start 2024-09-21T02:09:05.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 -- start start 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f149c0b41a0 0x7f149c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f149c156bd0 0x7f149c0c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.488+0000 7f14a0bc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 0x7f149c165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f14a0bc7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f149c169ee0 con 0x7f149c1601a0 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f14a0bc7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f149c169d60 con 0x7f149c0b41a0 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f14a0bc7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f149c16a060 con 0x7f149c156bd0 2024-09-21T02:09:05.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 0x7f149c165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 0x7f149c165580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51062/0 (socket says 172.21.15.96:51062) 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 -- 172.21.15.96:0/490686023 learned_addr learned my addr 172.21.15.96:0/490686023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1499d9b640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f149c156bd0 0x7f149c0c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149a59c640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f149c0b41a0 0x7f149c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 -- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f149c156bd0 msgr2=0x7f149c0c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f149c156bd0 0x7f149c0c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 -- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f149c0b41a0 msgr2=0x7f149c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f149c0b41a0 0x7f149c159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 -- 172.21.15.96:0/490686023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c157410 con 0x7f149c1601a0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f149ad9d640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 0x7f149c165580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f149400c3c0 tx=0x7f149400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940068a0 con 0x7f149c1601a0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1494006b30 con 0x7f149c1601a0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c0b7cf0 con 0x7f149c1601a0 2024-09-21T02:09:05.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494007020 con 0x7f149c1601a0 2024-09-21T02:09:05.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c15a230 con 0x7f149c1601a0 2024-09-21T02:09:05.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c0b7a40 con 0x7f149c1601a0 2024-09-21T02:09:05.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1494019020 con 0x7f149c1601a0 2024-09-21T02:09:05.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1492ffd640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f147c073200 0x7f147c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:05.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.492+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f1494093cb0 con 0x7f149c1601a0 2024-09-21T02:09:05.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.496+0000 7f149a59c640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f147c073200 0x7f147c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:05.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.496+0000 7f149a59c640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f147c073200 0x7f147c075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f148400b9a0 tx=0x7f1484005c30 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:05.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.500+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f149c0b4af0 con 0x7f149c1601a0 2024-09-21T02:09:05.532 INFO:tasks.ceph.mds.d.smithi122.stdout:starting mds.d at 2024-09-21T02:09:05.536 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-09-21T02:09:05.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.712+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f149c159930 con 0x7f149c1601a0 2024-09-21T02:09:05.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.712+0000 7f1492ffd640 1 -- 172.21.15.96:0/490686023 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f149405f100 con 0x7f149c1601a0 2024-09-21T02:09:05.716 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:05.716 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"btime":"2024-09-21T02:09:05:454632+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:09:05.716 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 2 2024-09-21T02:09:05.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.712+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f147c073200 msgr2=0x7f147c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.712+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f147c073200 0x7f147c075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f148400b9a0 tx=0x7f1484005c30 comp rx=0 tx=0).stop 2024-09-21T02:09:05.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.712+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 msgr2=0x7f149c165580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:05.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.712+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 0x7f149c165580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f149400c3c0 tx=0x7f149400c7c0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 shutdown_connections 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f147c073200 0x7f147c075710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f149c1601a0 0x7f149c165580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f149c156bd0 0x7f149c0c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 --2- 172.21.15.96:0/490686023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f149c0b41a0 0x7f149c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 >> 172.21.15.96:0/490686023 conn(0x7f149c0cf150 msgr2=0x7f149c0d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 shutdown_connections 2024-09-21T02:09:05.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:05.716+0000 7f14a0bc7640 1 -- 172.21.15.96:0/490686023 wait complete. 2024-09-21T02:09:07.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:07.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1c1b35640 1 -- 172.21.15.96:0/3782015844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a4004810 msgr2=0x7fd1a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:07.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/3782015844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a4004810 0x7fd1a4004c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009b60 tx=0x7fd1b001d290 comp rx=0 tx=0).stop 2024-09-21T02:09:07.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1bbfff640 1 -- 172.21.15.96:0/3782015844 reap_dead start 2024-09-21T02:09:07.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1c1b35640 1 -- 172.21.15.96:0/3782015844 shutdown_connections 2024-09-21T02:09:07.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1c1b35640 1 -- 172.21.15.96:0/3782015844 >> 172.21.15.96:0/3782015844 conn(0x7fd1bc0ccd70 msgr2=0x7fd1bc0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:07.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1c1b35640 1 -- 172.21.15.96:0/3782015844 shutdown_connections 2024-09-21T02:09:07.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.892+0000 7fd1c1b35640 1 -- 172.21.15.96:0/3782015844 wait complete. 2024-09-21T02:09:07.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 Processor -- start 2024-09-21T02:09:07.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 -- start start 2024-09-21T02:09:07.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1a4004810 0x7fd1bc0acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:07.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 0x7fd1bc0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1bc0cc8b0 0x7fd1bc147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ce740 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0cf2a0 con 0x7fd1a4004810 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ce8c0 con 0x7fd1bc0cc8b0 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 0x7fd1bc0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 0x7fd1bc0ad4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38402/0 (socket says 172.21.15.96:38402) 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 -- 172.21.15.96:0/660346823 learned_addr learned my addr 172.21.15.96:0/660346823 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1bb7fe640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1a4004810 0x7fd1bc0acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1bbfff640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1bc0cc8b0 0x7fd1bc147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 -- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1bc0cc8b0 msgr2=0x7fd1bc147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:07.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1bc0cc8b0 0x7fd1bc147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 -- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1a4004810 msgr2=0x7fd1bc0acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1a4004810 0x7fd1bc0acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 -- 172.21.15.96:0/660346823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc15dc70 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1bb7fe640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1a4004810 0x7fd1bc0acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1baffd640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 0x7fd1bc0ad4a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1b001d0c0 tx=0x7fd1b0005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00b1040 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1b00ac400 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc15e170 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0004290 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.896+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc15e730 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.900+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc15dee0 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.900+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1b00bd020 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.900+0000 7fd1a3fff640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd1a8073200 0x7fd1a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:07.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.900+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd1b012e630 con 0x7fd1bc1e4ea0 2024-09-21T02:09:07.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.900+0000 7fd1bb7fe640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd1a8073200 0x7fd1a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:07.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.900+0000 7fd1bb7fe640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd1a8073200 0x7fd1a8075710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac001000 tx=0x7fd1ac001ce0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:07.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:07.904+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd1b00c4020 con 0x7fd1bc1e4ea0 2024-09-21T02:09:08.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.100+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1bc06ce90 con 0x7fd1bc1e4ea0 2024-09-21T02:09:08.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.100+0000 7fd1a3fff640 1 -- 172.21.15.96:0/660346823 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7fd1b0016fa0 con 0x7fd1bc1e4ea0 2024-09-21T02:09:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:08.106 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"btime":"2024-09-21T02:09:06:456366+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:09:08.107 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-09-21T02:09:08.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd1a8073200 msgr2=0x7fd1a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd1a8073200 0x7fd1a8075710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac001000 tx=0x7fd1ac001ce0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 msgr2=0x7fd1bc0ad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 0x7fd1bc0ad4a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1b001d0c0 tx=0x7fd1b0005540 comp rx=0 tx=0).stop 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 shutdown_connections 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd1a8073200 0x7fd1a8075710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1bc0cc8b0 0x7fd1bc147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1bc1e4ea0 0x7fd1bc0ad4a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 --2- 172.21.15.96:0/660346823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1a4004810 0x7fd1bc0acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 >> 172.21.15.96:0/660346823 conn(0x7fd1bc0ccd70 msgr2=0x7fd1bc0b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 shutdown_connections 2024-09-21T02:09:08.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.104+0000 7fd1c1b35640 1 -- 172.21.15.96:0/660346823 wait complete. 2024-09-21T02:09:08.126 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:09:08.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 -- 172.21.15.96:0/248148492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a00004810 msgr2=0x7f3a00004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 --2- 172.21.15.96:0/248148492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a00004810 0x7f3a00004c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00c2e0 tx=0x7f39fc01cf20 comp rx=0 tx=0).stop 2024-09-21T02:09:08.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 -- 172.21.15.96:0/248148492 shutdown_connections 2024-09-21T02:09:08.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 --2- 172.21.15.96:0/248148492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a00004810 0x7f3a00004c20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 -- 172.21.15.96:0/248148492 >> 172.21.15.96:0/248148492 conn(0x7f3a1406eea0 msgr2=0x7f3a1406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:08.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 -- 172.21.15.96:0/248148492 shutdown_connections 2024-09-21T02:09:08.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.272+0000 7f3a1915a640 1 -- 172.21.15.96:0/248148492 wait complete. 2024-09-21T02:09:08.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 Processor -- start 2024-09-21T02:09:08.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- start start 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a00004810 0x7f3a14154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 0x7f3a141eb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a141ebe80 0x7f3a141fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3a140c4f10 con 0x7f3a14157b00 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a140c4d90 con 0x7f3a141ebe80 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3a140c5090 con 0x7f3a00004810 2024-09-21T02:09:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 0x7f3a141eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 0x7f3a141eb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38416/0 (socket says 172.21.15.96:38416) 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 -- 172.21.15.96:0/916980764 learned_addr learned my addr 172.21.15.96:0/916980764 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a12d9d640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a00004810 0x7f3a14154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1359e640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a141ebe80 0x7f3a141fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 -- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a00004810 msgr2=0x7f3a14154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a00004810 0x7f3a14154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 -- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a141ebe80 msgr2=0x7f3a141fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a141ebe80 0x7f3a141fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 -- 172.21.15.96:0/916980764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141ec290 con 0x7f3a14157b00 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0bfff640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 0x7f3a141eb940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f39fc01c9e0 tx=0x7f39fc006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0a5b80 con 0x7f3a14157b00 2024-09-21T02:09:08.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39fc0014c0 con 0x7f3a14157b00 2024-09-21T02:09:08.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0060d0 con 0x7f3a14157b00 2024-09-21T02:09:08.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141ea420 con 0x7f3a14157b00 2024-09-21T02:09:08.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141ea9e0 con 0x7f3a14157b00 2024-09-21T02:09:08.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.276+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a140cddb0 con 0x7f3a14157b00 2024-09-21T02:09:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.280+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f39fc006270 con 0x7f3a14157b00 2024-09-21T02:09:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.280+0000 7f3a0b7fe640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f39f4073230 0x7f39f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.280+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f39fc12d700 con 0x7f3a14157b00 2024-09-21T02:09:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.280+0000 7f3a12d9d640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f39f4073230 0x7f39f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.280+0000 7f3a12d9d640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f39f4073230 0x7f39f4075740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006fd0 tx=0x7f3a04006a90 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.284+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3a141ec290 con 0x7f3a14157b00 2024-09-21T02:09:08.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3a14073310 con 0x7f3a14157b00 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a0b7fe640 1 -- 172.21.15.96:0/916980764 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f39fc016cd0 con 0x7f3a14157b00 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:09:08.464 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-21T02:09:08.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f39f4073230 msgr2=0x7f39f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f39f4073230 0x7f39f4075740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006fd0 tx=0x7f3a04006a90 comp rx=0 tx=0).stop 2024-09-21T02:09:08.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 msgr2=0x7f3a141eb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 0x7f3a141eb940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f39fc01c9e0 tx=0x7f39fc006ca0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 shutdown_connections 2024-09-21T02:09:08.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f39f4073230 0x7f39f4075740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a141ebe80 0x7f3a141fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a14157b00 0x7f3a141eb940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 --2- 172.21.15.96:0/916980764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a00004810 0x7f3a14154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 >> 172.21.15.96:0/916980764 conn(0x7f3a1406eea0 msgr2=0x7f3a14160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:08.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 shutdown_connections 2024-09-21T02:09:08.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.460+0000 7f3a1915a640 1 -- 172.21.15.96:0/916980764 wait complete. 2024-09-21T02:09:08.484 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:09:08.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:09:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 -- 172.21.15.96:0/4097084705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3200cc870 msgr2=0x7fd3200cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 --2- 172.21.15.96:0/4097084705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3200cc870 0x7fd3200cf8f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd32006f170 tx=0x7fd31c01c810 comp rx=0 tx=0).stop 2024-09-21T02:09:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 -- 172.21.15.96:0/4097084705 shutdown_connections 2024-09-21T02:09:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 --2- 172.21.15.96:0/4097084705 >> v2:172.21.15.122:3300/0 conn(0x7fd32015ffc0 0x7fd3201603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 --2- 172.21.15.96:0/4097084705 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd32015c9d0 0x7fd32015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 --2- 172.21.15.96:0/4097084705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3200cc870 0x7fd3200cf8f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 -- 172.21.15.96:0/4097084705 >> 172.21.15.96:0/4097084705 conn(0x7fd3200d05a0 msgr2=0x7fd3200d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:08.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 -- 172.21.15.96:0/4097084705 shutdown_connections 2024-09-21T02:09:08.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 -- 172.21.15.96:0/4097084705 wait complete. 2024-09-21T02:09:08.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 Processor -- start 2024-09-21T02:09:08.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.640+0000 7fd327e5c640 1 -- start start 2024-09-21T02:09:08.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3200cc870 0x7fd320139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 0x7fd32015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd32015ffc0 0x7fd320073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd320070300 con 0x7fd32015c9d0 2024-09-21T02:09:08.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd320070180 con 0x7fd3200cc870 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd320070480 con 0x7fd32015ffc0 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 0x7fd32015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 0x7fd32015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38432/0 (socket says 172.21.15.96:38432) 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 -- 172.21.15.96:0/2679603973 learned_addr learned my addr 172.21.15.96:0/2679603973 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3263f9640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd32015ffc0 0x7fd320073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd325bf8640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3200cc870 0x7fd320139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 -- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd32015ffc0 msgr2=0x7fd320073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd32015ffc0 0x7fd320073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 -- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3200cc870 msgr2=0x7fd320139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3200cc870 0x7fd320139090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 -- 172.21.15.96:0/2679603973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd320074130 con 0x7fd32015c9d0 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd325bf8640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3200cc870 0x7fd320139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3253f7640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 0x7fd32015b420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd31000c970 tx=0x7fd31000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:08.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310007bb0 con 0x7fd32015c9d0 2024-09-21T02:09:08.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd310007e40 con 0x7fd32015c9d0 2024-09-21T02:09:08.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310005280 con 0x7fd32015c9d0 2024-09-21T02:09:08.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd320072da0 con 0x7fd32015c9d0 2024-09-21T02:09:08.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3200746d0 con 0x7fd32015c9d0 2024-09-21T02:09:08.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32015cde0 con 0x7fd32015c9d0 2024-09-21T02:09:08.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd310005420 con 0x7fd32015c9d0 2024-09-21T02:09:08.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3167fc640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd304073200 0x7fd304075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:08.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd3100942e0 con 0x7fd32015c9d0 2024-09-21T02:09:08.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.644+0000 7fd325bf8640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd304073200 0x7fd304075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:08.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.648+0000 7fd325bf8640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd304073200 0x7fd304075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0af000 tx=0x7fd31c01b050 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:08.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.652+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd310014030 con 0x7fd32015c9d0 2024-09-21T02:09:08.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:08.812+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd3200b1830 con 0x7fd32015c9d0 2024-09-21T02:09:09.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.476+0000 7fd3167fc640 1 -- 172.21.15.96:0/2679603973 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fd31005f6e0 con 0x7fd32015c9d0 2024-09-21T02:09:09.482 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd304073200 msgr2=0x7fd304075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd304073200 0x7fd304075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0af000 tx=0x7fd31c01b050 comp rx=0 tx=0).stop 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 msgr2=0x7fd32015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 0x7fd32015b420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd31000c970 tx=0x7fd31000ce30 comp rx=0 tx=0).stop 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 shutdown_connections 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd304073200 0x7fd304075710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd32015ffc0 0x7fd320073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd32015c9d0 0x7fd32015b420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 --2- 172.21.15.96:0/2679603973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3200cc870 0x7fd320139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 >> 172.21.15.96:0/2679603973 conn(0x7fd3200d05a0 msgr2=0x7fd3201560d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 shutdown_connections 2024-09-21T02:09:09.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.480+0000 7fd327e5c640 1 -- 172.21.15.96:0/2679603973 wait complete. 2024-09-21T02:09:09.514 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.668+0000 7fac41389640 1 -- 172.21.15.96:0/847520132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c15ee60 msgr2=0x7fac3c15f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.668+0000 7fac41389640 1 --2- 172.21.15.96:0/847520132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c15ee60 0x7fac3c15f270 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac3001c700 comp rx=0 tx=0).stop 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 -- 172.21.15.96:0/847520132 shutdown_connections 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 --2- 172.21.15.96:0/847520132 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c072c60 0x7fac3c073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 --2- 172.21.15.96:0/847520132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c15ee60 0x7fac3c15f270 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 -- 172.21.15.96:0/847520132 >> 172.21.15.96:0/847520132 conn(0x7fac3c0d0490 msgr2=0x7fac3c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 -- 172.21.15.96:0/847520132 shutdown_connections 2024-09-21T02:09:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 -- 172.21.15.96:0/847520132 wait complete. 2024-09-21T02:09:09.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 Processor -- start 2024-09-21T02:09:09.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.672+0000 7fac41389640 1 -- start start 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 0x7fac3c14ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c152300 0x7fac3c153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c15ee60 0x7fac3c1643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c168880 con 0x7fac3c072c60 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c168700 con 0x7fac3c15ee60 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c168a00 con 0x7fac3c152300 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 0x7fac3c14ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 0x7fac3c14ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38448/0 (socket says 172.21.15.96:38448) 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 -- 172.21.15.96:0/3974098610 learned_addr learned my addr 172.21.15.96:0/3974098610 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3a7fc640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c152300 0x7fac3c153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3b7fe640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c15ee60 0x7fac3c1643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 -- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c152300 msgr2=0x7fac3c153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c152300 0x7fac3c153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 -- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c15ee60 msgr2=0x7fac3c1643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c15ee60 0x7fac3c1643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 -- 172.21.15.96:0/3974098610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c1fbee0 con 0x7fac3c072c60 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3b7fe640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c15ee60 0x7fac3c1643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3a7fc640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c152300 0x7fac3c153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3affd640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 0x7fac3c14ce10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b410 tx=0x7fac2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c015030 con 0x7fac3c072c60 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac2c004020 con 0x7fac3c072c60 2024-09-21T02:09:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c1fc1c0 con 0x7fac3c072c60 2024-09-21T02:09:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0044b0 con 0x7fac3c072c60 2024-09-21T02:09:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1fc780 con 0x7fac3c072c60 2024-09-21T02:09:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c073070 con 0x7fac3c072c60 2024-09-21T02:09:09.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac2c018070 con 0x7fac3c072c60 2024-09-21T02:09:09.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac237fe640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fac28073230 0x7fac28075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:09.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7fac2c093f50 con 0x7fac3c072c60 2024-09-21T02:09:09.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.676+0000 7fac3a7fc640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fac28073230 0x7fac28075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:09.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.680+0000 7fac3a7fc640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fac28073230 0x7fac28075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fac300ae000 tx=0x7fac3001af40 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:09.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.680+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fac2c05f230 con 0x7fac3c072c60 2024-09-21T02:09:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:09.844+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fac3c068e50 con 0x7fac3c072c60 2024-09-21T02:09:10.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.492+0000 7fac237fe640 1 -- 172.21.15.96:0/3974098610 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7fac2c05eb30 con 0x7fac3c072c60 2024-09-21T02:09:10.497 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-21T02:09:10.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fac28073230 msgr2=0x7fac28075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:10.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fac28073230 0x7fac28075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fac300ae000 tx=0x7fac3001af40 comp rx=0 tx=0).stop 2024-09-21T02:09:10.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 msgr2=0x7fac3c14ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:10.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 0x7fac3c14ce10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b410 tx=0x7fac2c00b8d0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 shutdown_connections 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fac28073230 0x7fac28075740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c15ee60 0x7fac3c1643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac3c152300 0x7fac3c153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 --2- 172.21.15.96:0/3974098610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c072c60 0x7fac3c14ce10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 >> 172.21.15.96:0/3974098610 conn(0x7fac3c0d0490 msgr2=0x7fac3c165300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:10.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 shutdown_connections 2024-09-21T02:09:10.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.496+0000 7fac41389640 1 -- 172.21.15.96:0/3974098610 wait complete. 2024-09-21T02:09:10.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:09:10.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d7681640 1 -- 172.21.15.96:0/2676289560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75bc004b20 msgr2=0x7f75bc004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:10.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d7681640 1 --2- 172.21.15.96:0/2676289560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75bc004b20 0x7f75bc004f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00bb10 tx=0x7f75cc01cd80 comp rx=0 tx=0).stop 2024-09-21T02:09:10.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d5c1e640 1 -- 172.21.15.96:0/2676289560 reap_dead start 2024-09-21T02:09:10.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d7681640 1 -- 172.21.15.96:0/2676289560 shutdown_connections 2024-09-21T02:09:10.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d7681640 1 -- 172.21.15.96:0/2676289560 >> 172.21.15.96:0/2676289560 conn(0x7f75d006eea0 msgr2=0x7f75d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:10.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d7681640 1 -- 172.21.15.96:0/2676289560 shutdown_connections 2024-09-21T02:09:10.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.684+0000 7f75d7681640 1 -- 172.21.15.96:0/2676289560 wait complete. 2024-09-21T02:09:10.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 Processor -- start 2024-09-21T02:09:10.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 -- start start 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75d0154630 0x7f75d00c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0155a70 0x7f75d0155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 0x7f75d0145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00cbda0 con 0x7f75d0145560 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00cbc20 con 0x7f75d0155a70 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d7681640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f75d00cbf20 con 0x7f75d0154630 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 0x7f75d0145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 0x7f75d0145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38474/0 (socket says 172.21.15.96:38474) 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 -- 172.21.15.96:0/8993120 learned_addr learned my addr 172.21.15.96:0/8993120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:10.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d4c1c640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0155a70 0x7f75d0155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d541d640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75d0154630 0x7f75d00c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 -- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75d0154630 msgr2=0x7f75d00c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75d0154630 0x7f75d00c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 -- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0155a70 msgr2=0x7f75d0155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0155a70 0x7f75d0155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 -- 172.21.15.96:0/8993120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01463f0 con 0x7f75d0145560 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75d5c1e640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 0x7f75d0145970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75d0075670 tx=0x7f75c800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004b30 con 0x7f75d0145560 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75c8004dc0 con 0x7f75d0145560 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.688+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8003be0 con 0x7f75d0145560 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d0156550 con 0x7f75d0145560 2024-09-21T02:09:10.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d0156b10 con 0x7f75d0145560 2024-09-21T02:09:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d00cab60 con 0x7f75d0145560 2024-09-21T02:09:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f75c801b030 con 0x7f75d0145560 2024-09-21T02:09:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75c5ffb640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f75b4073230 0x7f75b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f75c8094960 con 0x7f75d0145560 2024-09-21T02:09:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75d541d640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f75b4073230 0x7f75b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.692+0000 7f75d541d640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f75b4073230 0x7f75b4075740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0b0000 tx=0x7f75cc0a7040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.696+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f75c8022020 con 0x7f75d0145560 2024-09-21T02:09:10.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:10.888+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f75d015b910 con 0x7f75d0145560 2024-09-21T02:09:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.536+0000 7f75c5ffb640 1 -- 172.21.15.96:0/8993120 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f75c805fbe0 con 0x7f75d0145560 2024-09-21T02:09:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f75b4073230 msgr2=0x7f75b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f75b4073230 0x7f75b4075740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0b0000 tx=0x7f75cc0a7040 comp rx=0 tx=0).stop 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 msgr2=0x7f75d0145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 0x7f75d0145970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75d0075670 tx=0x7f75c800ed50 comp rx=0 tx=0).stop 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 shutdown_connections 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f75b4073230 0x7f75b4075740 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d0145560 0x7f75d0145970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0155a70 0x7f75d0155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 --2- 172.21.15.96:0/8993120 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75d0154630 0x7f75d00c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:11.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 >> 172.21.15.96:0/8993120 conn(0x7f75d006eea0 msgr2=0x7f75d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:11.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 shutdown_connections 2024-09-21T02:09:11.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.544+0000 7f75d7681640 1 -- 172.21.15.96:0/8993120 wait complete. 2024-09-21T02:09:11.582 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T02:09:11.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T02:09:11.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f2566bce640 1 -- 172.21.15.96:0/1033504142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2548004e70 msgr2=0x7f25480052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:11.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f2566bce640 1 --2- 172.21.15.96:0/1033504142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2548004e70 0x7f25480052e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e00 tx=0x7f255801d150 comp rx=0 tx=0).stop 2024-09-21T02:09:11.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f256516b640 1 -- 172.21.15.96:0/1033504142 reap_dead start 2024-09-21T02:09:11.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f2566bce640 1 -- 172.21.15.96:0/1033504142 shutdown_connections 2024-09-21T02:09:11.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f2566bce640 1 -- 172.21.15.96:0/1033504142 >> 172.21.15.96:0/1033504142 conn(0x7f25600d05a0 msgr2=0x7f25600d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:11.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f2566bce640 1 -- 172.21.15.96:0/1033504142 shutdown_connections 2024-09-21T02:09:11.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.724+0000 7f2566bce640 1 -- 172.21.15.96:0/1033504142 wait complete. 2024-09-21T02:09:11.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 Processor -- start 2024-09-21T02:09:11.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- start start 2024-09-21T02:09:11.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25600ae0d0 0x7f25601f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:11.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25601f4ec0 0x7f256006f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:11.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 0x7f256006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:11.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2560164670 con 0x7f25601f52d0 2024-09-21T02:09:11.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25601644f0 con 0x7f25600ae0d0 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f25601647f0 con 0x7f25601f4ec0 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 0x7f256006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 0x7f256006fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38494/0 (socket says 172.21.15.96:38494) 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 -- 172.21.15.96:0/3542828655 learned_addr learned my addr 172.21.15.96:0/3542828655 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256496a640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25600ae0d0 0x7f25601f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f255ffff640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25601f4ec0 0x7f256006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 -- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25601f4ec0 msgr2=0x7f256006f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25601f4ec0 0x7f256006f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:11.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 -- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25600ae0d0 msgr2=0x7f25601f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25600ae0d0 0x7f25601f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 -- 172.21.15.96:0/3542828655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25601f9070 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f256516b640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 0x7f256006fc40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25580af040 tx=0x7f25580addc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580b1040 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25580bc070 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25601f9350 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558009a00 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25601f9910 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.728+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25600b36f0 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.732+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2558004a50 con 0x7f25601f52d0 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.732+0000 7f255d7fa640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f254c073200 0x7f254c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:11.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.732+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f255812f640 con 0x7f25601f52d0 2024-09-21T02:09:11.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.732+0000 7f256496a640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f254c073200 0x7f254c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:11.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.732+0000 7f256496a640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f254c073200 0x7f254c075710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2560159ac0 tx=0x7f2550008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:11.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.736+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25580c6020 con 0x7f25601f52d0 2024-09-21T02:09:11.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:11.900+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f2560159af0 con 0x7f25601f52d0 2024-09-21T02:09:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.500+0000 7f255d7fa640 1 -- 172.21.15.96:0/3542828655 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f2558016e60 con 0x7f25601f52d0 2024-09-21T02:09:12.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.500+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f254c073200 msgr2=0x7f254c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:12.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.500+0000 7f2566bce640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f254c073200 0x7f254c075710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2560159ac0 tx=0x7f2550008040 comp rx=0 tx=0).stop 2024-09-21T02:09:12.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 msgr2=0x7f256006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 0x7f256006fc40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25580af040 tx=0x7f25580addc0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 shutdown_connections 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f254c073200 0x7f254c075710 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25601f52d0 0x7f256006fc40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25601f4ec0 0x7f256006f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 --2- 172.21.15.96:0/3542828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25600ae0d0 0x7f25601f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 >> 172.21.15.96:0/3542828655 conn(0x7f25600d05a0 msgr2=0x7f256015a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 shutdown_connections 2024-09-21T02:09:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.504+0000 7f2566bce640 1 -- 172.21.15.96:0/3542828655 wait complete. 2024-09-21T02:09:12.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:09:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 -- 172.21.15.96:0/2258881627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 msgr2=0x7ffa740b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 --2- 172.21.15.96:0/2258881627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa740b34b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009bf0 tx=0x7ffa6c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 -- 172.21.15.96:0/2258881627 shutdown_connections 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 --2- 172.21.15.96:0/2258881627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa740b34b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 --2- 172.21.15.96:0/2258881627 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa7415f0a0 0x7ffa7415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 -- 172.21.15.96:0/2258881627 >> 172.21.15.96:0/2258881627 conn(0x7ffa7406eea0 msgr2=0x7ffa7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 -- 172.21.15.96:0/2258881627 shutdown_connections 2024-09-21T02:09:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.700+0000 7ffa789e0640 1 -- 172.21.15.96:0/2258881627 wait complete. 2024-09-21T02:09:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 Processor -- start 2024-09-21T02:09:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 -- start start 2024-09-21T02:09:12.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa74153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:12.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74152600 0x7ffa74153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:12.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 0x7ffa74153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74168af0 con 0x7ffa7415f0a0 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74168970 con 0x7ffa740b30a0 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa789e0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ffa74168c70 con 0x7ffa74152600 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 0x7ffa74153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 0x7ffa74153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38496/0 (socket says 172.21.15.96:38496) 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.704+0000 7ffa72d9d640 1 -- 172.21.15.96:0/1286675044 learned_addr learned my addr 172.21.15.96:0/1286675044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa7259c640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa74153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa71d9b640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74152600 0x7ffa74153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa72d9d640 1 -- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74152600 msgr2=0x7ffa74153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa72d9d640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74152600 0x7ffa74153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa72d9d640 1 -- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 msgr2=0x7ffa74153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa72d9d640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa74153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:12.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa72d9d640 1 -- 172.21.15.96:0/1286675044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa741426d0 con 0x7ffa7415f0a0 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa71d9b640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74152600 0x7ffa74153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa7259c640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa74153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa72d9d640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 0x7ffa74153ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c01b250 tx=0x7ffa6c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0b2040 con 0x7ffa7415f0a0 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa6c004a50 con 0x7ffa7415f0a0 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa74152d70 con 0x7ffa7415f0a0 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c004e60 con 0x7ffa7415f0a0 2024-09-21T02:09:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa74151220 con 0x7ffa7415f0a0 2024-09-21T02:09:12.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa740b34b0 con 0x7ffa7415f0a0 2024-09-21T02:09:12.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa6c0be060 con 0x7ffa7415f0a0 2024-09-21T02:09:12.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa62ffd640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ffa54073200 0x7ffa54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:12.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7ffa6c12df30 con 0x7ffa7415f0a0 2024-09-21T02:09:12.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.708+0000 7ffa7259c640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ffa54073200 0x7ffa54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:12.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.712+0000 7ffa7259c640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ffa54073200 0x7ffa54075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa640046c0 tx=0x7ffa64009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.712+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffa6c016790 con 0x7ffa7415f0a0 2024-09-21T02:09:12.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:12.888+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffa74166000 con 0x7ffa7415f0a0 2024-09-21T02:09:14.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.556+0000 7ffa62ffd640 1 -- 172.21.15.96:0/1286675044 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7ffa6c0f8e20 con 0x7ffa7415f0a0 2024-09-21T02:09:14.563 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data_ec' created 2024-09-21T02:09:14.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ffa54073200 msgr2=0x7ffa54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ffa54073200 0x7ffa54075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa640046c0 tx=0x7ffa64009040 comp rx=0 tx=0).stop 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 msgr2=0x7ffa74153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 0x7ffa74153ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c01b250 tx=0x7ffa6c009200 comp rx=0 tx=0).stop 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 shutdown_connections 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ffa54073200 0x7ffa54075710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa7415f0a0 0x7ffa74153ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffa74152600 0x7ffa74153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 --2- 172.21.15.96:0/1286675044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa740b30a0 0x7ffa74153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.560+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 >> 172.21.15.96:0/1286675044 conn(0x7ffa7406eea0 msgr2=0x7ffa74151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.564+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 shutdown_connections 2024-09-21T02:09:14.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.564+0000 7ffa789e0640 1 -- 172.21.15.96:0/1286675044 wait complete. 2024-09-21T02:09:14.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 -- 172.21.15.96:0/4153854755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c15ba70 msgr2=0x7fe07c15be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 --2- 172.21.15.96:0/4153854755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c15ba70 0x7fe07c15be80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0726b0 tx=0x7fe07401cc60 comp rx=0 tx=0).stop 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 -- 172.21.15.96:0/4153854755 shutdown_connections 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 --2- 172.21.15.96:0/4153854755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c15ba70 0x7fe07c15be80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 --2- 172.21.15.96:0/4153854755 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe07c15f060 0x7fe07c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 --2- 172.21.15.96:0/4153854755 >> v2:172.21.15.122:3300/0 conn(0x7fe07c0b2ef0 0x7fe07c0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 -- 172.21.15.96:0/4153854755 >> 172.21.15.96:0/4153854755 conn(0x7fe07c0d05a0 msgr2=0x7fe07c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 -- 172.21.15.96:0/4153854755 shutdown_connections 2024-09-21T02:09:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 -- 172.21.15.96:0/4153854755 wait complete. 2024-09-21T02:09:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 Processor -- start 2024-09-21T02:09:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.752+0000 7fe082efb640 1 -- start start 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 0x7fe07c1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c15ba70 0x7fe07c13e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe07c15f060 0x7fe07c165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c074050 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c073ed0 con 0x7fe07c15ba70 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c0741d0 con 0x7fe07c15f060 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 0x7fe07c1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 0x7fe07c1f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38508/0 (socket says 172.21.15.96:38508) 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 -- 172.21.15.96:0/4098423618 learned_addr learned my addr 172.21.15.96:0/4098423618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 -- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe07c15f060 msgr2=0x7fe07c165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe07c15f060 0x7fe07c165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 -- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c15ba70 msgr2=0x7fe07c13e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c15ba70 0x7fe07c13e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 -- 172.21.15.96:0/4098423618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c141a90 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe080c97640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 0x7fe07c1f71a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00d700 tx=0x7fe06c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c017070 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe06c009b10 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c1411f0 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c010460 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c1417b0 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.756+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c15be80 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.760+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe06c01a070 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.760+0000 7fe0797fa640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe068073200 0x7fe068075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:14.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.760+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7fe06c094a70 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.760+0000 7fe07bfff640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe068073200 0x7fe068075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:14.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.760+0000 7fe07bfff640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe068073200 0x7fe068075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0700058d0 tx=0x7fe070005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.764+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe06c05f770 con 0x7fe07c0b2ef0 2024-09-21T02:09:14.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:14.936+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe07c070660 con 0x7fe07c0b2ef0 2024-09-21T02:09:15.466 INFO:tasks.ceph.mds.c.smithi096.stderr:2024-09-21T02:09:15.460+0000 7efd5ad08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:09:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.560+0000 7fe0797fa640 1 -- 172.21.15.96:0/4098423618 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7fe06c05f070 con 0x7fe07c0b2ef0 2024-09-21T02:09:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe068073200 msgr2=0x7fe068075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe068073200 0x7fe068075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0700058d0 tx=0x7fe070005860 comp rx=0 tx=0).stop 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 msgr2=0x7fe07c1f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 0x7fe07c1f71a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00d700 tx=0x7fe06c00dbc0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 shutdown_connections 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe068073200 0x7fe068075710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe07c15f060 0x7fe07c165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c15ba70 0x7fe07c13e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 --2- 172.21.15.96:0/4098423618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe07c0b2ef0 0x7fe07c1f71a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 >> 172.21.15.96:0/4098423618 conn(0x7fe07c0d05a0 msgr2=0x7fe07c15ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 shutdown_connections 2024-09-21T02:09:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.564+0000 7fe082efb640 1 -- 172.21.15.96:0/4098423618 wait complete. 2024-09-21T02:09:15.606 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T02:09:15.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 -- 172.21.15.96:0/1151412529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9406e5b0 msgr2=0x7fbe9415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:15.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 --2- 172.21.15.96:0/1151412529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9406e5b0 0x7fbe9415bea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe940cfe00 tx=0x7fbe8801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 -- 172.21.15.96:0/1151412529 shutdown_connections 2024-09-21T02:09:15.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 --2- 172.21.15.96:0/1151412529 >> v2:172.21.15.122:3300/0 conn(0x7fbe9415c3e0 0x7fbe940cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 --2- 172.21.15.96:0/1151412529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9406e5b0 0x7fbe9415bea0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 -- 172.21.15.96:0/1151412529 >> 172.21.15.96:0/1151412529 conn(0x7fbe9406eea0 msgr2=0x7fbe9406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:15.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 -- 172.21.15.96:0/1151412529 shutdown_connections 2024-09-21T02:09:15.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.764+0000 7fbe99744640 1 -- 172.21.15.96:0/1151412529 wait complete. 2024-09-21T02:09:15.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 Processor -- start 2024-09-21T02:09:15.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- start start 2024-09-21T02:09:15.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe940ced00 0x7fbe94167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:15.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 0x7fbe9414c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:15.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe941fc3b0 0x7fbe9414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94166850 con 0x7fbe9415c3e0 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941666d0 con 0x7fbe941fc3b0 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbe941669d0 con 0x7fbe940ced00 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 0x7fbe9414c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 0x7fbe9414c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38522/0 (socket says 172.21.15.96:38522) 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 -- 172.21.15.96:0/2954869922 learned_addr learned my addr 172.21.15.96:0/2954869922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe937fe640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe941fc3b0 0x7fbe9414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe92ffd640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe940ced00 0x7fbe94167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 -- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe940ced00 msgr2=0x7fbe94167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe940ced00 0x7fbe94167aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 -- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe941fc3b0 msgr2=0x7fbe9414f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:15.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe941fc3b0 0x7fbe9414f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 -- 172.21.15.96:0/2954869922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe94168660 con 0x7fbe9415c3e0 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe937fe640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe941fc3b0 0x7fbe9414f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe927fc640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 0x7fbe9414c350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe880b0040 tx=0x7fbe880a5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe880b2040 con 0x7fbe9415c3e0 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe880043b0 con 0x7fbe9415c3e0 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe94168380 con 0x7fbe9415c3e0 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe880052d0 con 0x7fbe9415c3e0 2024-09-21T02:09:15.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe94154710 con 0x7fbe9415c3e0 2024-09-21T02:09:15.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.768+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94154240 con 0x7fbe9415c3e0 2024-09-21T02:09:15.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.772+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe88005470 con 0x7fbe9415c3e0 2024-09-21T02:09:15.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.772+0000 7fbe7b7fe640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbe80073200 0x7fbe80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:15.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.772+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7fbe8812d940 con 0x7fbe9415c3e0 2024-09-21T02:09:15.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.772+0000 7fbe92ffd640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbe80073200 0x7fbe80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:15.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.772+0000 7fbe92ffd640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbe80073200 0x7fbe80075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a750 tx=0x7fbe84008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:15.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.776+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbe880164b0 con 0x7fbe9415c3e0 2024-09-21T02:09:15.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:15.988+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fbe94166f70 con 0x7fbe9415c3e0 2024-09-21T02:09:16.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.580+0000 7fbe7b7fe640 1 -- 172.21.15.96:0/2954869922 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fbe880f86e0 con 0x7fbe9415c3e0 2024-09-21T02:09:16.584 INFO:teuthology.orchestra.run.smithi096.stderr:added data pool 4 to fsmap 2024-09-21T02:09:16.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbe80073200 msgr2=0x7fbe80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbe80073200 0x7fbe80075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a750 tx=0x7fbe84008040 comp rx=0 tx=0).stop 2024-09-21T02:09:16.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 msgr2=0x7fbe9414c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 0x7fbe9414c350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe880b0040 tx=0x7fbe880a5540 comp rx=0 tx=0).stop 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 shutdown_connections 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbe80073200 0x7fbe80075710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe941fc3b0 0x7fbe9414f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9415c3e0 0x7fbe9414c350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 --2- 172.21.15.96:0/2954869922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe940ced00 0x7fbe94167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 >> 172.21.15.96:0/2954869922 conn(0x7fbe9406eea0 msgr2=0x7fbe9406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 shutdown_connections 2024-09-21T02:09:16.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.584+0000 7fbe99744640 1 -- 172.21.15.96:0/2954869922 wait complete. 2024-09-21T02:09:16.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:16.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 -- 172.21.15.96:0/1273234378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e00d2b70 msgr2=0x7f10e00d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/1273234378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e00d2b70 0x7f10e00d2f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009840 tx=0x7f10d801c820 comp rx=0 tx=0).stop 2024-09-21T02:09:16.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 -- 172.21.15.96:0/1273234378 shutdown_connections 2024-09-21T02:09:16.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/1273234378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e00d2b70 0x7f10e00d2f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 -- 172.21.15.96:0/1273234378 >> 172.21.15.96:0/1273234378 conn(0x7f10e00d1b10 msgr2=0x7f10e00d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:16.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 -- 172.21.15.96:0/1273234378 shutdown_connections 2024-09-21T02:09:16.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 -- 172.21.15.96:0/1273234378 wait complete. 2024-09-21T02:09:16.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 Processor -- start 2024-09-21T02:09:16.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.772+0000 7f10e5aa8640 1 -- start start 2024-09-21T02:09:16.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f10e00cf710 0x7f10e0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:16.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e00d2b70 0x7f10e00cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:16.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 0x7f10e0157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:16.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10e016d2b0 con 0x7f10e015f9a0 2024-09-21T02:09:16.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10e016d130 con 0x7f10e00d2b70 2024-09-21T02:09:16.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f10e016d430 con 0x7f10e00cf710 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 0x7f10e0157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 0x7f10e0157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38550/0 (socket says 172.21.15.96:38550) 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 -- 172.21.15.96:0/3474703335 learned_addr learned my addr 172.21.15.96:0/3474703335 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10df7fe640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f10e00cf710 0x7f10e0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10deffd640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e00d2b70 0x7f10e00cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 -- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f10e00cf710 msgr2=0x7f10e0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f10e00cf710 0x7f10e0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 -- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e00d2b70 msgr2=0x7f10e00cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e00d2b70 0x7f10e00cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 -- 172.21.15.96:0/3474703335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e01f9b30 con 0x7f10e015f9a0 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10dffff640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 0x7f10e0157e90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009840 tx=0x7f10d80a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80b2040 con 0x7f10e015f9a0 2024-09-21T02:09:16.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d80bb070 con 0x7f10e015f9a0 2024-09-21T02:09:16.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e01f8d70 con 0x7f10e015f9a0 2024-09-21T02:09:16.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8003600 con 0x7f10e015f9a0 2024-09-21T02:09:16.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e01f9330 con 0x7f10e015f9a0 2024-09-21T02:09:16.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e00d2f80 con 0x7f10e015f9a0 2024-09-21T02:09:16.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f10d80037a0 con 0x7f10e015f9a0 2024-09-21T02:09:16.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10d7fff640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f10c0073200 0x7f10c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:16.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.776+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f10d812eb00 con 0x7f10e015f9a0 2024-09-21T02:09:16.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.780+0000 7f10df7fe640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f10c0073200 0x7f10c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:16.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.780+0000 7f10df7fe640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f10c0073200 0x7f10c0075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10e00b7610 tx=0x7f10d0008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:16.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.784+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f10d8016410 con 0x7f10e015f9a0 2024-09-21T02:09:16.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.992+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10e0156630 con 0x7f10e015f9a0 2024-09-21T02:09:16.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.992+0000 7f10d7fff640 1 -- 172.21.15.96:0/3474703335 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5225 (secure 0 0 0) 0x7f10d80f97e0 con 0x7f10e015f9a0 2024-09-21T02:09:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:16.997 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"btime":"2024-09-21T02:09:16:579791+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:15.990655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:09:16.997 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 7 2024-09-21T02:09:16.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f10c0073200 msgr2=0x7f10c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f10c0073200 0x7f10c0075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10e00b7610 tx=0x7f10d0008040 comp rx=0 tx=0).stop 2024-09-21T02:09:16.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 msgr2=0x7f10e0157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:16.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 0x7f10e0157e90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009840 tx=0x7f10d80a5700 comp rx=0 tx=0).stop 2024-09-21T02:09:16.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 shutdown_connections 2024-09-21T02:09:16.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f10c0073200 0x7f10c0075710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e015f9a0 0x7f10e0157e90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e00d2b70 0x7f10e00cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 --2- 172.21.15.96:0/3474703335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f10e00cf710 0x7f10e0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 >> 172.21.15.96:0/3474703335 conn(0x7f10e00d1b10 msgr2=0x7f10e0151900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 shutdown_connections 2024-09-21T02:09:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:16.996+0000 7f10e5aa8640 1 -- 172.21.15.96:0/3474703335 wait complete. 2024-09-21T02:09:17.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 -- 172.21.15.96:0/2148752299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d240cec00 msgr2=0x7f5d240cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/2148752299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d240cec00 0x7f5d240cf010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d24074bb0 tx=0x7f5d1001c830 comp rx=0 tx=0).stop 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 -- 172.21.15.96:0/2148752299 shutdown_connections 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/2148752299 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5d240c4c90 0x7f5d240c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/2148752299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d240cec00 0x7f5d240cf010 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 -- 172.21.15.96:0/2148752299 >> 172.21.15.96:0/2148752299 conn(0x7f5d2406eea0 msgr2=0x7f5d2406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 -- 172.21.15.96:0/2148752299 shutdown_connections 2024-09-21T02:09:17.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.160+0000 7f5d2aed3640 1 -- 172.21.15.96:0/2148752299 wait complete. 2024-09-21T02:09:17.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 Processor -- start 2024-09-21T02:09:17.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- start start 2024-09-21T02:09:17.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 0x7f5d240cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5d240c4c90 0x7f5d24155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d240cec00 0x7f5d240addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24169c60 con 0x7f5d2406e2c0 2024-09-21T02:09:17.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24169ae0 con 0x7f5d240cec00 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5d24169de0 con 0x7f5d240c4c90 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 0x7f5d240cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 0x7f5d240cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42562/0 (socket says 172.21.15.96:42562) 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 -- 172.21.15.96:0/1914599430 learned_addr learned my addr 172.21.15.96:0/1914599430 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d23fff640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5d240c4c90 0x7f5d24155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 -- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5d240c4c90 msgr2=0x7f5d24155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5d240c4c90 0x7f5d24155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 -- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d240cec00 msgr2=0x7f5d240addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d240cec00 0x7f5d240addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 -- 172.21.15.96:0/1914599430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d240cb520 con 0x7f5d2406e2c0 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d28c6f640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 0x7f5d240cb790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800b440 tx=0x7f5d1800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:17.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18015020 con 0x7f5d2406e2c0 2024-09-21T02:09:17.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d18011980 con 0x7f5d2406e2c0 2024-09-21T02:09:17.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d24151f00 con 0x7f5d2406e2c0 2024-09-21T02:09:17.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18011dd0 con 0x7f5d2406e2c0 2024-09-21T02:09:17.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d241524c0 con 0x7f5d2406e2c0 2024-09-21T02:09:17.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.164+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d240c50a0 con 0x7f5d2406e2c0 2024-09-21T02:09:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.168+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d18018020 con 0x7f5d2406e2c0 2024-09-21T02:09:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.168+0000 7f5d217fa640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5d08073200 0x7f5d08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.168+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5d18094460 con 0x7f5d2406e2c0 2024-09-21T02:09:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.168+0000 7f5d23fff640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5d08073200 0x7f5d08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.168+0000 7f5d23fff640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5d08073200 0x7f5d08075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5d100ae000 tx=0x7f5d1001b070 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:17.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.172+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5d1801f030 con 0x7f5d2406e2c0 2024-09-21T02:09:17.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.332+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d24073310 con 0x7f5d2406e2c0 2024-09-21T02:09:17.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.332+0000 7f5d217fa640 1 -- 172.21.15.96:0/1914599430 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7f5d1800d040 con 0x7f5d2406e2c0 2024-09-21T02:09:17.337 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:17.337 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:17.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5d08073200 msgr2=0x7f5d08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5d08073200 0x7f5d08075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5d100ae000 tx=0x7f5d1001b070 comp rx=0 tx=0).stop 2024-09-21T02:09:17.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 msgr2=0x7f5d240cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 0x7f5d240cb790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800b440 tx=0x7f5d1800b900 comp rx=0 tx=0).stop 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 shutdown_connections 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5d08073200 0x7f5d08075710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d240cec00 0x7f5d240addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5d240c4c90 0x7f5d24155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 --2- 172.21.15.96:0/1914599430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d2406e2c0 0x7f5d240cb790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 >> 172.21.15.96:0/1914599430 conn(0x7f5d2406eea0 msgr2=0x7f5d24153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 shutdown_connections 2024-09-21T02:09:17.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.336+0000 7f5d2aed3640 1 -- 172.21.15.96:0/1914599430 wait complete. 2024-09-21T02:09:17.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:17.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 -- 172.21.15.96:0/3420903732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 msgr2=0x7f920c15db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 --2- 172.21.15.96:0/3420903732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15db90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f920c0747a0 tx=0x7f91fc01c8a0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 -- 172.21.15.96:0/3420903732 shutdown_connections 2024-09-21T02:09:17.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 --2- 172.21.15.96:0/3420903732 >> v2:172.21.15.122:3300/0 conn(0x7f920c0741e0 0x7f920c15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 --2- 172.21.15.96:0/3420903732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15db90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 --2- 172.21.15.96:0/3420903732 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f920c0d0900 0x7f920c073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 -- 172.21.15.96:0/3420903732 >> 172.21.15.96:0/3420903732 conn(0x7f920c070720 msgr2=0x7f920c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:17.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 -- 172.21.15.96:0/3420903732 shutdown_connections 2024-09-21T02:09:17.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.500+0000 7f9210a76640 1 -- 172.21.15.96:0/3420903732 wait complete. 2024-09-21T02:09:17.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 Processor -- start 2024-09-21T02:09:17.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 -- start start 2024-09-21T02:09:17.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920c0741e0 0x7f920c1463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f920c0d0900 0x7f920c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1606d0 con 0x7f920c15cfc0 2024-09-21T02:09:17.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f920c160550 con 0x7f920c0741e0 2024-09-21T02:09:17.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f920c160850 con 0x7f920c0d0900 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42572/0 (socket says 172.21.15.96:42572) 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 -- 172.21.15.96:0/80550949 learned_addr learned my addr 172.21.15.96:0/80550949 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920a59c640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920c0741e0 0x7f920c1463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9209d9b640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f920c0d0900 0x7f920c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 -- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f920c0d0900 msgr2=0x7f920c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f920c0d0900 0x7f920c1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 -- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920c0741e0 msgr2=0x7f920c1463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920c0741e0 0x7f920c1463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 -- 172.21.15.96:0/80550949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c0b0bf0 con 0x7f920c15cfc0 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f920ad9d640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15e950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009850 tx=0x7f91fc0a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:17.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0bb070 con 0x7f920c15cfc0 2024-09-21T02:09:17.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91fc0043b0 con 0x7f920c15cfc0 2024-09-21T02:09:17.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0b2040 con 0x7f920c15cfc0 2024-09-21T02:09:17.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c146910 con 0x7f920c15cfc0 2024-09-21T02:09:17.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.504+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c158d20 con 0x7f920c15cfc0 2024-09-21T02:09:17.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.508+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c160d40 con 0x7f920c15cfc0 2024-09-21T02:09:17.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.508+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91fc0be070 con 0x7f920c15cfc0 2024-09-21T02:09:17.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.508+0000 7f91faffd640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f91ec073230 0x7f91ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.508+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f91fc12e000 con 0x7f920c15cfc0 2024-09-21T02:09:17.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.508+0000 7f920a59c640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f91ec073230 0x7f91ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.508+0000 7f920a59c640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f91ec073230 0x7f91ec075740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9204005c50 tx=0x7f9204005be0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:17.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.512+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f91fc0f3290 con 0x7f920c15cfc0 2024-09-21T02:09:17.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.672+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f920c0ce660 con 0x7f920c15cfc0 2024-09-21T02:09:17.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.676+0000 7f91faffd640 1 -- 172.21.15.96:0/80550949 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7f91fc016490 con 0x7f920c15cfc0 2024-09-21T02:09:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:17.680 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.676+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f91ec073230 msgr2=0x7f91ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.676+0000 7f9210a76640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f91ec073230 0x7f91ec075740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9204005c50 tx=0x7f9204005be0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.676+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 msgr2=0x7f920c15e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.676+0000 7f9210a76640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15e950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009850 tx=0x7f91fc0a5790 comp rx=0 tx=0).stop 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 shutdown_connections 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f91ec073230 0x7f91ec075740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c15cfc0 0x7f920c15e950 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f920c0d0900 0x7f920c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 --2- 172.21.15.96:0/80550949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920c0741e0 0x7f920c1463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 >> 172.21.15.96:0/80550949 conn(0x7f920c070720 msgr2=0x7f920c14a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 shutdown_connections 2024-09-21T02:09:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.680+0000 7f9210a76640 1 -- 172.21.15.96:0/80550949 wait complete. 2024-09-21T02:09:17.714 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:17.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 -- 172.21.15.96:0/2126255738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800cf710 msgr2=0x7fd68015f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 --2- 172.21.15.96:0/2126255738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800cf710 0x7fd68015f290 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd68006ecf0 tx=0x7fd67c01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 -- 172.21.15.96:0/2126255738 shutdown_connections 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 --2- 172.21.15.96:0/2126255738 >> v2:172.21.15.122:3300/0 conn(0x7fd6800d3470 0x7fd680162b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 --2- 172.21.15.96:0/2126255738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800cf710 0x7fd68015f290 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 -- 172.21.15.96:0/2126255738 >> 172.21.15.96:0/2126255738 conn(0x7fd6800d1b10 msgr2=0x7fd6800d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 -- 172.21.15.96:0/2126255738 shutdown_connections 2024-09-21T02:09:17.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 -- 172.21.15.96:0/2126255738 wait complete. 2024-09-21T02:09:17.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 Processor -- start 2024-09-21T02:09:17.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.852+0000 7fd6887b5640 1 -- start start 2024-09-21T02:09:17.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6800cf710 0x7fd6800b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 0x7fd680151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd68015f7d0 0x7fd680151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd68016bdb0 con 0x7fd6800d3470 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd68015fdf0 con 0x7fd6800cf710 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd68016bf30 con 0x7fd68015f7d0 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 0x7fd680151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 0x7fd680151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42582/0 (socket says 172.21.15.96:42582) 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 -- 172.21.15.96:0/3155335336 learned_addr learned my addr 172.21.15.96:0/3155335336 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd686551640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6800cf710 0x7fd6800b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd686d52640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd68015f7d0 0x7fd680151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 -- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd68015f7d0 msgr2=0x7fd680151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd68015f7d0 0x7fd680151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 -- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6800cf710 msgr2=0x7fd6800b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6800cf710 0x7fd6800b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 -- 172.21.15.96:0/3155335336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6800b4640 con 0x7fd6800d3470 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd686551640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6800cf710 0x7fd6800b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd685d50640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 0x7fd680151200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd67000b3e0 tx=0x7fd67000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670015020 con 0x7fd6800d3470 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd670011940 con 0x7fd6800d3470 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670011d90 con 0x7fd6800d3470 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6800b4920 con 0x7fd6800d3470 2024-09-21T02:09:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6800cd560 con 0x7fd6800d3470 2024-09-21T02:09:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6800b43b0 con 0x7fd6800d3470 2024-09-21T02:09:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd670018020 con 0x7fd6800d3470 2024-09-21T02:09:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd676ffd640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd664073200 0x7fd664075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.856+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd6700944a0 con 0x7fd6800d3470 2024-09-21T02:09:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.860+0000 7fd686551640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd664073200 0x7fd664075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.860+0000 7fd686551640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd664073200 0x7fd664075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0af000 tx=0x7fd67c01b100 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:17.864+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd67001f020 con 0x7fd6800d3470 2024-09-21T02:09:18.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.064+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd68015f290 con 0x7fd6800d3470 2024-09-21T02:09:18.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.064+0000 7fd676ffd640 1 -- 172.21.15.96:0/3155335336 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5225 (secure 0 0 0) 0x7fd67005f210 con 0x7fd6800d3470 2024-09-21T02:09:18.070 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:18.071 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"btime":"2024-09-21T02:09:16:579791+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:15.990655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:09:18.071 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 7 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd664073200 msgr2=0x7fd664075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd664073200 0x7fd664075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0af000 tx=0x7fd67c01b100 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 msgr2=0x7fd680151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 0x7fd680151200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd67000b3e0 tx=0x7fd67000b8a0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 shutdown_connections 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd664073200 0x7fd664075710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd68015f7d0 0x7fd680151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6800d3470 0x7fd680151200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 --2- 172.21.15.96:0/3155335336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6800cf710 0x7fd6800b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 >> 172.21.15.96:0/3155335336 conn(0x7fd6800d1b10 msgr2=0x7fd680160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 shutdown_connections 2024-09-21T02:09:18.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.068+0000 7fd6887b5640 1 -- 172.21.15.96:0/3155335336 wait complete. 2024-09-21T02:09:18.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 -- 172.21.15.96:0/3126952574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efda4004810 msgr2=0x7efda4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/3126952574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efda4004810 0x7efda4004c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efda00016f0 tx=0x7efda001c790 comp rx=0 tx=0).stop 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 -- 172.21.15.96:0/3126952574 shutdown_connections 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/3126952574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efda4004810 0x7efda4004c20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 -- 172.21.15.96:0/3126952574 >> 172.21.15.96:0/3126952574 conn(0x7efdb806eea0 msgr2=0x7efdb806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 -- 172.21.15.96:0/3126952574 shutdown_connections 2024-09-21T02:09:18.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.224+0000 7efdbd3ca640 1 -- 172.21.15.96:0/3126952574 wait complete. 2024-09-21T02:09:18.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 Processor -- start 2024-09-21T02:09:18.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 -- start start 2024-09-21T02:09:18.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efda4004810 0x7efdb8158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 0x7efdb8154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb81552f0 0x7efdb8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efdb815c020 con 0x7efdb8158b90 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efdb815bea0 con 0x7efda4004810 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efdb815c1a0 con 0x7efdb81552f0 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 0x7efdb8154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 0x7efdb8154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42594/0 (socket says 172.21.15.96:42594) 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 -- 172.21.15.96:0/810338101 learned_addr learned my addr 172.21.15.96:0/810338101 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdb6ffd640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efda4004810 0x7efdb8158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 -- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb81552f0 msgr2=0x7efdb8154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdb77fe640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb81552f0 0x7efdb8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb81552f0 0x7efdb8154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 -- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efda4004810 msgr2=0x7efdb8158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efda4004810 0x7efdb8158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 -- 172.21.15.96:0/810338101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81f1260 con 0x7efdb8158b90 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdb6ffd640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efda4004810 0x7efdb8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaffff640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 0x7efdb8154770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efda001b060 tx=0x7efda0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00bd070 con 0x7efdb8158b90 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efda0006500 con 0x7efdb8158b90 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81f1500 con 0x7efdb8158b90 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00a6730 con 0x7efdb8158b90 2024-09-21T02:09:18.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.228+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81f1ac0 con 0x7efdb8158b90 2024-09-21T02:09:18.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.232+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb8075b80 con 0x7efdb8158b90 2024-09-21T02:09:18.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.232+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efda00a6ae0 con 0x7efdb8158b90 2024-09-21T02:09:18.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.232+0000 7efdaf7fe640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efd98073230 0x7efd98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.232+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efda012eaa0 con 0x7efdb8158b90 2024-09-21T02:09:18.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.232+0000 7efdb6ffd640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efd98073230 0x7efd98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.232+0000 7efdb6ffd640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efd98073230 0x7efd98075740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efda8001a40 tx=0x7efda8000f60 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.236+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efda00aa050 con 0x7efdb8158b90 2024-09-21T02:09:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.396+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdb806df80 con 0x7efdb8158b90 2024-09-21T02:09:18.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.396+0000 7efdaf7fe640 1 -- 172.21.15.96:0/810338101 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7efda00164f0 con 0x7efdb8158b90 2024-09-21T02:09:18.402 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:18.403 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efd98073230 msgr2=0x7efd98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efd98073230 0x7efd98075740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efda8001a40 tx=0x7efda8000f60 comp rx=0 tx=0).stop 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 msgr2=0x7efdb8154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 0x7efdb8154770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efda001b060 tx=0x7efda0003b70 comp rx=0 tx=0).stop 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 shutdown_connections 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efd98073230 0x7efd98075740 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb81552f0 0x7efdb8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdb8158b90 0x7efdb8154770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 --2- 172.21.15.96:0/810338101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efda4004810 0x7efdb8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 >> 172.21.15.96:0/810338101 conn(0x7efdb806eea0 msgr2=0x7efdb80cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 shutdown_connections 2024-09-21T02:09:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.400+0000 7efdbd3ca640 1 -- 172.21.15.96:0/810338101 wait complete. 2024-09-21T02:09:18.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:09:10.893814+0000', 'modified': '2024-09-21T02:09:15.990655+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4353}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4353': {'gid': 4353, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/2262930956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2262930956}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2262930956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4353, 'qdb_cluster': [4353]} 2024-09-21T02:09:18.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:09:18.825 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:09:18.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 -- 172.21.15.96:0/2806393485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7040c4c90 msgr2=0x7fa7040c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/2806393485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7040c4c90 0x7fa7040c50a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa704074bb0 tx=0x7fa6fc01ccf0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 -- 172.21.15.96:0/2806393485 shutdown_connections 2024-09-21T02:09:18.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/2806393485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7040c4c90 0x7fa7040c50a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/2806393485 >> v2:172.21.15.122:3300/0 conn(0x7fa7040cec00 0x7fa7040cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 -- 172.21.15.96:0/2806393485 >> 172.21.15.96:0/2806393485 conn(0x7fa70406eea0 msgr2=0x7fa70406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 -- 172.21.15.96:0/2806393485 shutdown_connections 2024-09-21T02:09:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 -- 172.21.15.96:0/2806393485 wait complete. 2024-09-21T02:09:18.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 Processor -- start 2024-09-21T02:09:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.984+0000 7fa70c7e5640 1 -- start start 2024-09-21T02:09:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 0x7fa7041562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7040c4c90 0x7fa704161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7040cec00 0x7fa704160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa704169c60 con 0x7fa70406e2c0 2024-09-21T02:09:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa704169ae0 con 0x7fa7040cec00 2024-09-21T02:09:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa704169de0 con 0x7fa7040c4c90 2024-09-21T02:09:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 0x7fa7041562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 0x7fa7041562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42610/0 (socket says 172.21.15.96:42610) 2024-09-21T02:09:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 -- 172.21.15.96:0/274466889 learned_addr learned my addr 172.21.15.96:0/274466889 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70ad82640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7040cec00 0x7fa704160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa709d80640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7040c4c90 0x7fa704161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 -- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7040c4c90 msgr2=0x7fa704161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7040c4c90 0x7fa704161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 -- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7040cec00 msgr2=0x7fa704160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7040cec00 0x7fa704160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 -- 172.21.15.96:0/274466889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa704156800 con 0x7fa70406e2c0 2024-09-21T02:09:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70a581640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 0x7fa7041562c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa70000b410 tx=0x7fa70000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa6faffd640 1 -- 172.21.15.96:0/274466889 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700015020 con 0x7fa70406e2c0 2024-09-21T02:09:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa6faffd640 1 -- 172.21.15.96:0/274466889 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa700011920 con 0x7fa70406e2c0 2024-09-21T02:09:18.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7040c9f90 con 0x7fa70406e2c0 2024-09-21T02:09:18.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa6faffd640 1 -- 172.21.15.96:0/274466889 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700011d70 con 0x7fa70406e2c0 2024-09-21T02:09:18.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7040c9270 con 0x7fa70406e2c0 2024-09-21T02:09:19.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.988+0000 7fa6faffd640 1 -- 172.21.15.96:0/274466889 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa700018020 con 0x7fa70406e2c0 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa6faffd640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa6e8073230 0x7fa6e8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa6faffd640 1 -- 172.21.15.96:0/274466889 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa700093fa0 con 0x7fa70406e2c0 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa709d80640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa6e8073230 0x7fa6e8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 Processor -- start 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- start start 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa709d80640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa6e8073230 0x7fa6e8075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa704167290 tx=0x7fa6f4008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c67c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c52b950 0x55ff8c67cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c67d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff8c551410 con 0x55ff8c13b460 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ff8c48eb40 con 0x55ff8c514ca0 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55ff8c47df70 con 0x55ff8c52b950 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c67c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c67c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42614/0 (socket says 172.21.15.96:42614) 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 -- 172.21.15.96:0/2020537899 learned_addr learned my addr 172.21.15.96:0/2020537899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:19.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa709d80640 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c52b950 0x55ff8c67cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70ad82640 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c67d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 -- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c52b950 msgr2=0x55ff8c67cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c52b950 0x55ff8c67cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 -- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 msgr2=0x55ff8c67d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c67d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 -- 172.21.15.96:0/2020537899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff8c4455b0 con 0x55ff8c13b460 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa70a581640 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c67c6a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa700013000 tx=0x7fa70005e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2193b6ed4b4ecd3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa6e7fff640 1 -- 172.21.15.96:0/2020537899 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700078030 con 0x55ff8c13b460 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa6e7fff640 1 -- 172.21.15.96:0/2020537899 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa700063060 con 0x55ff8c13b460 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa6e7fff640 1 -- 172.21.15.96:0/2020537899 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700067d70 con 0x55ff8c13b460 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- 172.21.15.96:0/2020537899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 msgr2=0x55ff8c67c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c67c6a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa700013000 tx=0x7fa70005e1d0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- 172.21.15.96:0/2020537899 shutdown_connections 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c67d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c52b950 0x55ff8c67cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2020537899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c67c6a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- 172.21.15.96:0/2020537899 >> 172.21.15.96:0/2020537899 conn(0x55ff8c601d90 msgr2=0x55ff8c631690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.992+0000 7fa713ab0000 1 -- 172.21.15.96:0/2020537899 shutdown_connections 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- 172.21.15.96:0/2020537899 wait complete. 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- start start 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 Processor -- start 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c13b460 0x55ff8c683920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c683e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 0x55ff8c6843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff8c4185f0 con 0x55ff8c52b950 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ff8c481d80 con 0x55ff8c514ca0 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55ff8c424520 con 0x55ff8c13b460 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 10 client.0 _ll_register_callbacks cb 0x55ff8c428830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:19.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 0x55ff8c6843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 0x55ff8c6843a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42622/0 (socket says 172.21.15.96:42622) 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 -- 172.21.15.96:0/913894277 learned_addr learned my addr 172.21.15.96:0/913894277 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70a581640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c13b460 0x55ff8c683920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa709d80640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c683e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 -- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c13b460 msgr2=0x55ff8c683920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c13b460 0x55ff8c683920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 -- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 msgr2=0x55ff8c683e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c683e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 -- 172.21.15.96:0/913894277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff8c41c190 con 0x55ff8c52b950 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70ad82640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 0x55ff8c6843a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc01b3a0 tx=0x7fa6fc0a5a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5b59177b00901f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa6e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa6e57fa640 1 -- 172.21.15.96:0/913894277 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0b2040 con 0x55ff8c52b950 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa6e57fa640 1 -- 172.21.15.96:0/913894277 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa6fc004c50 con 0x55ff8c52b950 2024-09-21T02:09:19.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff8c416110 con 0x55ff8c52b950 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa6e57fa640 1 -- 172.21.15.96:0/913894277 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa6fc0c4050 con 0x55ff8c52b950 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 10 client.4476 fetch_fsmap learned FSMap version 7 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ff8c502f10 con 0x55ff8c52b950 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa6e57fa640 1 -- 172.21.15.96:0/913894277 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ff8c416110 con 0x55ff8c52b950 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa6e57fa640 1 -- 172.21.15.96:0/913894277 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7fa6fc0c7020 con 0x55ff8c52b950 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 10 client.4476 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 10 client.4476 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 20 client.4476 populate_metadata read hostname 'smithi096' 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c1207d0 0x55ff8c68dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 4 client.4476 mds_command: new command op to 4353 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 --> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff8c490a90 con 0x55ff8c1207d0 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:18.996+0000 7fa70a581640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c1207d0 0x55ff8c68dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa6e57fa640 10 client.4476 ms_handle_connect on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa70a581640 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c1207d0 0x55ff8c68dd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929eef72b67b722f server_cookie=3a63620bbbe73aa9 in_seq=0 out_seq=0 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa6e57fa640 1 -- 172.21.15.96:0/913894277 <== mds.0 v2:172.21.15.96:6834/2262930956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ff8c490a90 con 0x55ff8c1207d0 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa6e57fa640 10 client.4476 handle_command_reply: tid=0 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 client.4476 shutdown 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 msgr2=0x55ff8c6843a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 0x55ff8c6843a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc01b3a0 tx=0x7fa6fc0a5a30 comp rx=0 tx=0).stop 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 shutdown_connections 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c1207d0 0x55ff8c68dd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c52b950 0x55ff8c6843a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c683e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa6e57fa640 0 client.4476 ms_handle_reset on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 --2- 172.21.15.96:0/913894277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c13b460 0x55ff8c683920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 >> 172.21.15.96:0/913894277 conn(0x55ff8c601d90 msgr2=0x55ff8c638440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 shutdown_connections 2024-09-21T02:09:19.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 1 -- 172.21.15.96:0/913894277 wait complete. 2024-09-21T02:09:19.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.000+0000 7fa713ab0000 20 client.4476 trim_cache size 0 max 16384 2024-09-21T02:09:19.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.016+0000 7fa713ab0000 1 Processor -- start 2024-09-21T02:09:19.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.016+0000 7fa713ab0000 1 -- start start 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c6eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa704204400 con 0x55ff8c13b460 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa704203460 con 0x55ff8c514ca0 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa7042027c0 con 0x55ff8c601d90 2024-09-21T02:09:19.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c6eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c6eeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42630/0 (socket says 172.21.15.96:42630) 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 -- 172.21.15.96:0/195743164 learned_addr learned my addr 172.21.15.96:0/195743164 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa70ad82640 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa70a581640 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 -- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 msgr2=0x55ff8c6ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 -- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 msgr2=0x55ff8c6ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ef0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 -- 172.21.15.96:0/195743164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff8c4d9db0 con 0x55ff8c13b460 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa709d80640 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c6eeb60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c8c0 tx=0x7fa6f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d1c247f912197 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa6e77fe640 1 -- 172.21.15.96:0/195743164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4013b90 con 0x55ff8c13b460 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa6e77fe640 1 -- 172.21.15.96:0/195743164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f4024070 con 0x55ff8c13b460 2024-09-21T02:09:19.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa6e77fe640 1 -- 172.21.15.96:0/195743164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f401f690 con 0x55ff8c13b460 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- 172.21.15.96:0/195743164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 msgr2=0x55ff8c6eeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c6eeb60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c8c0 tx=0x7fa6f4013000 comp rx=0 tx=0).stop 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- 172.21.15.96:0/195743164 shutdown_connections 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c13b460 0x55ff8c6eeb60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 --2- 172.21.15.96:0/195743164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- 172.21.15.96:0/195743164 >> 172.21.15.96:0/195743164 conn(0x55ff8c638740 msgr2=0x55ff8c67a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- 172.21.15.96:0/195743164 shutdown_connections 2024-09-21T02:09:19.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- 172.21.15.96:0/195743164 wait complete. 2024-09-21T02:09:19.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 -- start start 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.020+0000 7fa713ab0000 1 Processor -- start 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c13b460 0x55ff8c6ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff8c481090 con 0x55ff8c514ca0 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ff8c449ee0 con 0x55ff8c13b460 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55ff8c53d750 con 0x55ff8c601d90 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 10 client.0 _ll_register_callbacks cb 0x7fa6fc0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42646/0 (socket says 172.21.15.96:42646) 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 -- 172.21.15.96:0/2078900112 learned_addr learned my addr 172.21.15.96:0/2078900112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70a581640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c13b460 0x55ff8c6ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 -- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 msgr2=0x55ff8c6aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 -- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c13b460 msgr2=0x55ff8c6ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c13b460 0x55ff8c6ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 -- 172.21.15.96:0/2078900112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff8c444ab0 con 0x55ff8c514ca0 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70a581640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c13b460 0x55ff8c6ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa709d80640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ae760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55ff8c45adb0 tx=0x7fa6f400cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf7b165d560d7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-21T02:09:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 1 -- 172.21.15.96:0/2078900112 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f401f630 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 1 -- 172.21.15.96:0/2078900112 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa6f401f7d0 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f4024070 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 1 -- 172.21.15.96:0/2078900112 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa6f4039050 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 10 client.4482 fetch_fsmap learned FSMap version 7 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ff8c423070 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 1 -- 172.21.15.96:0/2078900112 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f4024070 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 1 -- 172.21.15.96:0/2078900112 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7fa6f403c020 con 0x55ff8c514ca0 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 10 client.4482 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 10 client.4482 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 20 client.4482 populate_metadata read hostname 'smithi096' 2024-09-21T02:09:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c139e80 0x55ff8c6fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 4 client.4482 mds_command: new command op to 4353 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 --> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] -- command(tid 0: {"prefix": "status"}) -- 0x55ff8c48c540 con 0x55ff8c139e80 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70a581640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c139e80 0x55ff8c6fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 10 client.4482 ms_handle_connect on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70a581640 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c139e80 0x55ff8c6fa6a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de1bb23de03ab43d server_cookie=27882b7c9b9df356 in_seq=0 out_seq=0 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 1 -- 172.21.15.96:0/2078900112 <== mds.0 v2:172.21.15.96:6834/2262930956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+316 (crc 0 0 0) 0x55ff8c48c540 con 0x55ff8c139e80 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 10 client.4482 handle_command_reply: tid=0 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 client.4482 shutdown 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 msgr2=0x55ff8c6ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ae760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55ff8c45adb0 tx=0x7fa6f400cfb0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 shutdown_connections 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x55ff8c139e80 0x55ff8c6fa6a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55ff8c601d90 0x55ff8c6aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff8c514ca0 0x55ff8c6ae760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 --2- 172.21.15.96:0/2078900112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff8c13b460 0x55ff8c6ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa6e57fa640 0 client.4482 ms_handle_reset on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 >> 172.21.15.96:0/2078900112 conn(0x55ff8c638740 msgr2=0x55ff8c67afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 shutdown_connections 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 1 -- 172.21.15.96:0/2078900112 wait complete. 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa713ab0000 20 client.4482 trim_cache size 0 max 16384 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "892bf3fa-0a6f-4242-b0ca-536a4d478630", 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4353, 2024-09-21T02:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 7.486559958, 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 7, 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 26, 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 25, 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 13.58772327 2024-09-21T02:09:19.037 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:09:19.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa6e8073230 msgr2=0x7fa6e8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa6e8073230 0x7fa6e8075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa704167290 tx=0x7fa6f4008040 comp rx=0 tx=0).stop 2024-09-21T02:09:19.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 msgr2=0x7fa7041562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 0x7fa7041562c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa70000b410 tx=0x7fa70000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 shutdown_connections 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa6e8073230 0x7fa6e8075740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7040cec00 0x7fa704160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7040c4c90 0x7fa704161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 --2- 172.21.15.96:0/274466889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa70406e2c0 0x7fa7041562c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 >> 172.21.15.96:0/274466889 conn(0x7fa70406eea0 msgr2=0x7fa70406fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 shutdown_connections 2024-09-21T02:09:19.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.024+0000 7fa70c7e5640 1 -- 172.21.15.96:0/274466889 wait complete. 2024-09-21T02:09:19.048 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:09:19.048 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:19.048 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:09:19.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 -- 172.21.15.96:0/3120560245 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec15c5b0 msgr2=0x7f52ec0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3120560245 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec15c5b0 0x7f52ec0cbaf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009bc0 tx=0x7f52e401cba0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 -- 172.21.15.96:0/3120560245 shutdown_connections 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3120560245 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec15c5b0 0x7f52ec0cbaf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3120560245 >> v2:172.21.15.122:3300/0 conn(0x7f52ec0ced00 0x7f52ec06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 -- 172.21.15.96:0/3120560245 >> 172.21.15.96:0/3120560245 conn(0x7f52ec06eea0 msgr2=0x7f52ec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 -- 172.21.15.96:0/3120560245 shutdown_connections 2024-09-21T02:09:19.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.180+0000 7f52f32d5640 1 -- 172.21.15.96:0/3120560245 wait complete. 2024-09-21T02:09:19.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 Processor -- start 2024-09-21T02:09:19.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 -- start start 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 0x7f52ec1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec0ced00 0x7f52ec0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ec15c5b0 0x7f52ec14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec1686c0 con 0x7f52ec06e3e0 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec168540 con 0x7f52ec15c5b0 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52ec168840 con 0x7f52ec0ced00 2024-09-21T02:09:19.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 0x7f52ec1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 0x7f52ec1556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42662/0 (socket says 172.21.15.96:42662) 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 -- 172.21.15.96:0/3250614010 learned_addr learned my addr 172.21.15.96:0/3250614010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f0870640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec0ced00 0x7f52ec0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1872640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ec15c5b0 0x7f52ec14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 -- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec0ced00 msgr2=0x7f52ec0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec0ced00 0x7f52ec0ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 -- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ec15c5b0 msgr2=0x7f52ec14c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ec15c5b0 0x7f52ec14c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 -- 172.21.15.96:0/3250614010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec14ca20 con 0x7f52ec06e3e0 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1872640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ec15c5b0 0x7f52ec14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f0870640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec0ced00 0x7f52ec0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:19.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f1071640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 0x7f52ec1556d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52e800ea20 tx=0x7f52e800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e800cd80 con 0x7f52ec06e3e0 2024-09-21T02:09:19.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52e8004020 con 0x7f52ec06e3e0 2024-09-21T02:09:19.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ec152010 con 0x7f52ec06e3e0 2024-09-21T02:09:19.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8004450 con 0x7f52ec06e3e0 2024-09-21T02:09:19.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.184+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ec154ed0 con 0x7f52ec06e3e0 2024-09-21T02:09:19.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.188+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ec06dea0 con 0x7f52ec06e3e0 2024-09-21T02:09:19.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.188+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f52e8003600 con 0x7f52ec06e3e0 2024-09-21T02:09:19.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.188+0000 7f52ddffb640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f52d0073200 0x7f52d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:19.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.188+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f52e8095040 con 0x7f52ec06e3e0 2024-09-21T02:09:19.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.188+0000 7f52f0870640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f52d0073200 0x7f52d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:19.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.188+0000 7f52f0870640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f52d0073200 0x7f52d0075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52ec1525d0 tx=0x7f52e0009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:19.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.192+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f52e805fca0 con 0x7f52ec06e3e0 2024-09-21T02:09:19.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.384+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52ec0af610 con 0x7f52ec06e3e0 2024-09-21T02:09:19.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52ddffb640 1 -- 172.21.15.96:0/3250614010 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f52e805f5a0 con 0x7f52ec06e3e0 2024-09-21T02:09:19.391 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T02:09:19.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f52d0073200 msgr2=0x7f52d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f52d0073200 0x7f52d0075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52ec1525d0 tx=0x7f52e0009040 comp rx=0 tx=0).stop 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 msgr2=0x7f52ec1556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 0x7f52ec1556d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52e800ea20 tx=0x7f52e800ee20 comp rx=0 tx=0).stop 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 shutdown_connections 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f52d0073200 0x7f52d0075710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ec15c5b0 0x7f52ec14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ec0ced00 0x7f52ec0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 --2- 172.21.15.96:0/3250614010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ec06e3e0 0x7f52ec1556d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 >> 172.21.15.96:0/3250614010 conn(0x7f52ec06eea0 msgr2=0x7f52ec15d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 shutdown_connections 2024-09-21T02:09:19.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19.388+0000 7f52f32d5640 1 -- 172.21.15.96:0/3250614010 wait complete. 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T02:09:19.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:09:19.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:09:19.412 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T02:09:19.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:19.412 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-21T02:09:19.418 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:09:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42493sec preferred_lft 42493sec 2024-09-21T02:09:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-21T02:09:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:19.420 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:09:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:e1:97:6e:59 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:09:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:19.420 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:09:19.421 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:19.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:19.535 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:09:19.587 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-21T02:09:19.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:19.588 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-21T02:09:19.834 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:19.835 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:19.836 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:19.836 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:09:19.836 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:19.836 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:19.836 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:09:19.836 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:09:19.836 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:09:19.837 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:19.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:19.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:19.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:19.917 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-21T02:09:19.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:19.964 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-09-21T02:09:20.011 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:20.011 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:20.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:20.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T02:09:20.011 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:20.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:20.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:20.123 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T02:09:20.124 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:20.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:20.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:20.286 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:20.286 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:20.286 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-21T02:09:20.287 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:09:20.287 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:20.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:20.389 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T02:09:20.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:20.389 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:20.434 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T02:09:20.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:20.435 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:20.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:20.538 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:20.589 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:20.590 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:20.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:20.591 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:20.640 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:09:20.641 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T02:09:20.684 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:20.698 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:20.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:20.751 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:20.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi096.stderr:2024-09-21T02:09:20.792+0000 7ff20920b580 -1 init, newargv = 0x55efd15fd590 newargc=15 2024-09-21T02:09:20.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi096.stderr:ceph-fuse[35449]: starting ceph client 2024-09-21T02:09:20.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi096.stderr:ceph-fuse[35449]: starting fuse 2024-09-21T02:09:21.902 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:21.917 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:21.964 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:21.964 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:21.964 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:21.967 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:21.967 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:21.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:21.967 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:22.015 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-21T02:09:22.016 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T02:09:22.016 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:22.017 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.admin")) 2024-09-21T02:09:22.018 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-21T02:09:22.145 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.admin.35449.asok 2024-09-21T02:09:22.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:22.157 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.35449.asok 2024-09-21T02:09:22.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:22.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.35449.asok status 2024-09-21T02:09:22.341 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:09:22.341 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-21T02:09:22.341 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "admin", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "35449", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4491, 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-21T02:09:22.342 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4491 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 698700314 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 698700314 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4491 192.168.0.1:0/698700314", 2024-09-21T02:09:22.343 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/698700314", 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 7, 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 26, 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-21T02:09:22.344 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:09:22.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:22.357 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-21T02:09:22.357 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:22.363 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-21T02:09:22.363 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:22.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:22.364 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:22.419 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T02:09:22.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:09:22.546 INFO:teuthology.orchestra.run.smithi096.stdout:# file: . 2024-09-21T02:09:22.546 INFO:teuthology.orchestra.run.smithi096.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T02:09:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:22.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:22.553 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:09:22.553 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:09:22.558 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:09:22.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- 172.21.15.96:0/1209620833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d80ced00 msgr2=0x7fb6d806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:22.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/1209620833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d80ced00 0x7fb6d806dea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80b61b0 tx=0x7fb6d401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- 172.21.15.96:0/1209620833 shutdown_connections 2024-09-21T02:09:22.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/1209620833 >> v2:172.21.15.122:3300/0 conn(0x7fb6d806e3e0 0x7fb6d815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/1209620833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d80ced00 0x7fb6d806dea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- 172.21.15.96:0/1209620833 >> 172.21.15.96:0/1209620833 conn(0x7fb6d806eea0 msgr2=0x7fb6d806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:22.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- 172.21.15.96:0/1209620833 shutdown_connections 2024-09-21T02:09:22.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- 172.21.15.96:0/1209620833 wait complete. 2024-09-21T02:09:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 Processor -- start 2024-09-21T02:09:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- start start 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6d806e3e0 0x7fb6d80b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6d815c3e0 0x7fb6d80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 0x7fb6d80ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8168bf0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d80cf330 con 0x7fb6d815c3e0 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6dfac4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d8168d70 con 0x7fb6d806e3e0 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6de061640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 0x7fb6d80ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6de061640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 0x7fb6d80ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42678/0 (socket says 172.21.15.96:42678) 2024-09-21T02:09:22.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.756+0000 7fb6de061640 1 -- 172.21.15.96:0/3740012087 learned_addr learned my addr 172.21.15.96:0/3740012087 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6de061640 1 -- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6d806e3e0 msgr2=0x7fb6d80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6de061640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6d806e3e0 0x7fb6d80b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6de061640 1 -- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6d815c3e0 msgr2=0x7fb6d80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6de061640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6d815c3e0 0x7fb6d80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6de061640 1 -- 172.21.15.96:0/3740012087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d80b2ef0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6de061640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 0x7fb6d80ce130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000c6c0 tx=0x7fb6d000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0003b80 con 0x7fb6d81f8a30 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6d0003d40 con 0x7fb6d81f8a30 2024-09-21T02:09:22.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0005330 con 0x7fb6d81f8a30 2024-09-21T02:09:22.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d80b0ca0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d81f9490 con 0x7fb6d81f8a30 2024-09-21T02:09:22.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d806dea0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb6d00054d0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6ce7fc640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb6bc073200 0x7fb6bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:22.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb6d0094280 con 0x7fb6d81f8a30 2024-09-21T02:09:22.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.760+0000 7fb6dd860640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb6bc073200 0x7fb6bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:22.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.764+0000 7fb6dd860640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb6bc073200 0x7fb6bc075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40af000 tx=0x7fb6d4019660 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.768+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb6d806e7f0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.952+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb6d80725f0 con 0x7fb6d81f8a30 2024-09-21T02:09:22.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.952+0000 7fb6ce7fc640 1 -- 172.21.15.96:0/3740012087 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fb6d005f080 con 0x7fb6d81f8a30 2024-09-21T02:09:22.958 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:09:22.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb6bc073200 msgr2=0x7fb6bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:22.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb6bc073200 0x7fb6bc075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40af000 tx=0x7fb6d4019660 comp rx=0 tx=0).stop 2024-09-21T02:09:22.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 msgr2=0x7fb6d80ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:22.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 0x7fb6d80ce130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000c6c0 tx=0x7fb6d000cb80 comp rx=0 tx=0).stop 2024-09-21T02:09:22.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 shutdown_connections 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb6bc073200 0x7fb6bc075710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6d81f8a30 0x7fb6d80ce130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6d815c3e0 0x7fb6d80cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 --2- 172.21.15.96:0/3740012087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6d806e3e0 0x7fb6d80b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 >> 172.21.15.96:0/3740012087 conn(0x7fb6d806eea0 msgr2=0x7fb6d806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 shutdown_connections 2024-09-21T02:09:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:22.956+0000 7fb6dfac4640 1 -- 172.21.15.96:0/3740012087 wait complete. 2024-09-21T02:09:22.979 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:09:22.979 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:09:22.985 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:09:23.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61a3fff640 1 -- 172.21.15.96:0/3333022778 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a40ad440 con 0x7f61ac0cc360 2024-09-21T02:09:23.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 -- 172.21.15.96:0/3333022778 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac0cc360 msgr2=0x7f61ac0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:23.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 --2- 172.21.15.96:0/3333022778 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac0cc360 0x7f61ac0cf420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009bc0 tx=0x7f61a401cb10 comp rx=0 tx=0).stop 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 -- 172.21.15.96:0/3333022778 shutdown_connections 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 --2- 172.21.15.96:0/3333022778 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac0cc360 0x7f61ac0cf420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 --2- 172.21.15.96:0/3333022778 >> v2:172.21.15.122:3300/0 conn(0x7f61ac155870 0x7f61ac06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 -- 172.21.15.96:0/3333022778 >> 172.21.15.96:0/3333022778 conn(0x7f61ac073d10 msgr2=0x7f61ac074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 -- 172.21.15.96:0/3333022778 shutdown_connections 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.176+0000 7f61b36fd640 1 -- 172.21.15.96:0/3333022778 wait complete. 2024-09-21T02:09:23.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 Processor -- start 2024-09-21T02:09:23.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 -- start start 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac06f4f0 0x7f61ac14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 0x7f61ac154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac155870 0x7f61ac0b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac16ba10 con 0x7f61ac0cc360 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac16b890 con 0x7f61ac155870 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f61ac16bb90 con 0x7f61ac06f4f0 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 0x7f61ac154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:23.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 0x7f61ac154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42680/0 (socket says 172.21.15.96:42680) 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 -- 172.21.15.96:0/728472442 learned_addr learned my addr 172.21.15.96:0/728472442 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b1c9a640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac155870 0x7f61ac0b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b1499640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac06f4f0 0x7f61ac14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 -- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac06f4f0 msgr2=0x7f61ac14dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac06f4f0 0x7f61ac14dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 -- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac155870 msgr2=0x7f61ac0b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac155870 0x7f61ac0b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 -- 172.21.15.96:0/728472442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac1513b0 con 0x7f61ac0cc360 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b0c98640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 0x7f61ac154590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f619c00c3c0 tx=0x7f619c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c017070 con 0x7f61ac0cc360 2024-09-21T02:09:23.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f619c0053c0 con 0x7f61ac0cc360 2024-09-21T02:09:23.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac0b4aa0 con 0x7f61ac0cc360 2024-09-21T02:09:23.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c003860 con 0x7f61ac0cc360 2024-09-21T02:09:23.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.180+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac145080 con 0x7f61ac0cc360 2024-09-21T02:09:23.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.184+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac14d5a0 con 0x7f61ac0cc360 2024-09-21T02:09:23.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.184+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f619c01a070 con 0x7f61ac0cc360 2024-09-21T02:09:23.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.184+0000 7f61a1ffb640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6190073230 0x7f6190075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:23.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.184+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f619c093850 con 0x7f61ac0cc360 2024-09-21T02:09:23.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.184+0000 7f61b1499640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6190073230 0x7f6190075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:23.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.184+0000 7f61b1499640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6190073230 0x7f6190075740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ad0 tx=0x7f61a8009260 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:23.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.188+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f61ac06f900 con 0x7f61ac0cc360 2024-09-21T02:09:23.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.356+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f61ac06e970 con 0x7f61ac0cc360 2024-09-21T02:09:23.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.356+0000 7f61a1ffb640 1 -- 172.21.15.96:0/728472442 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f619c05def0 con 0x7f61ac0cc360 2024-09-21T02:09:23.361 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6190073230 msgr2=0x7f6190075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6190073230 0x7f6190075740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ad0 tx=0x7f61a8009260 comp rx=0 tx=0).stop 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 msgr2=0x7f61ac154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 0x7f61ac154590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f619c00c3c0 tx=0x7f619c00c880 comp rx=0 tx=0).stop 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 shutdown_connections 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6190073230 0x7f6190075740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac155870 0x7f61ac0b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac0cc360 0x7f61ac154590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 --2- 172.21.15.96:0/728472442 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61ac06f4f0 0x7f61ac14dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 >> 172.21.15.96:0/728472442 conn(0x7f61ac073d10 msgr2=0x7f61ac0cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 shutdown_connections 2024-09-21T02:09:23.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:23.360+0000 7f61b36fd640 1 -- 172.21.15.96:0/728472442 wait complete. 2024-09-21T02:09:23.381 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-21T02:09:23.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:23.382 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T02:09:23.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi096.stderr:ceph-fuse[35449]: fuse finished with error 0 and tester_r 0 2024-09-21T02:09:23.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:09:29.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:09:29.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:29.509 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T02:09:29.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 -- 172.21.15.96:0/2383141266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f532006e2c0 msgr2=0x7f53200c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 --2- 172.21.15.96:0/2383141266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f532006e2c0 0x7f53200c4580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5320074bb0 tx=0x7f531c01caf0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 -- 172.21.15.96:0/2383141266 shutdown_connections 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 --2- 172.21.15.96:0/2383141266 >> v2:172.21.15.122:3300/0 conn(0x7f53200c4ac0 0x7f53200c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 --2- 172.21.15.96:0/2383141266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f532006e2c0 0x7f53200c4580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 -- 172.21.15.96:0/2383141266 >> 172.21.15.96:0/2383141266 conn(0x7f532006eea0 msgr2=0x7f532006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 -- 172.21.15.96:0/2383141266 shutdown_connections 2024-09-21T02:09:29.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 -- 172.21.15.96:0/2383141266 wait complete. 2024-09-21T02:09:29.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 Processor -- start 2024-09-21T02:09:29.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.724+0000 7f532838b640 1 -- start start 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f532006e2c0 0x7f53201504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 0x7f53200c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f53200cdbf0 0x7f5320156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53200cf0a0 con 0x7f53200c4ac0 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53200cef20 con 0x7f532006e2c0 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f53200cf220 con 0x7f53200cdbf0 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 0x7f53200c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 0x7f53200c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59640/0 (socket says 172.21.15.96:59640) 2024-09-21T02:09:29.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 -- 172.21.15.96:0/1713970161 learned_addr learned my addr 172.21.15.96:0/1713970161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5326127640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f532006e2c0 0x7f53201504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5326928640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f53200cdbf0 0x7f5320156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 -- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f53200cdbf0 msgr2=0x7f5320156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f53200cdbf0 0x7f5320156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 -- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f532006e2c0 msgr2=0x7f53201504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f532006e2c0 0x7f53201504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 -- 172.21.15.96:0/1713970161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53200c4350 con 0x7f53200c4ac0 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5326127640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f532006e2c0 0x7f53201504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5325926640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 0x7f53200c3e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53100058d0 tx=0x7f531000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310017070 con 0x7f53200c4ac0 2024-09-21T02:09:29.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5310009d20 con 0x7f53200c4ac0 2024-09-21T02:09:29.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5320150d00 con 0x7f53200c4ac0 2024-09-21T02:09:29.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53100106c0 con 0x7f53200c4ac0 2024-09-21T02:09:29.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201512c0 con 0x7f53200c4ac0 2024-09-21T02:09:29.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.728+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53200c4580 con 0x7f53200c4ac0 2024-09-21T02:09:29.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.732+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f531001a070 con 0x7f53200c4ac0 2024-09-21T02:09:29.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.732+0000 7f5316ffd640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5304073200 0x7f5304075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:29.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.732+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f53100946a0 con 0x7f53200c4ac0 2024-09-21T02:09:29.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.732+0000 7f5326127640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5304073200 0x7f5304075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:29.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.732+0000 7f5326127640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5304073200 0x7f5304075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f531c0af000 tx=0x7f531c01b330 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:29.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.736+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f531005f410 con 0x7f53200c4ac0 2024-09-21T02:09:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.900+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5320150a90 con 0x7f53200c4ac0 2024-09-21T02:09:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f5316ffd640 1 -- 172.21.15.96:0/1713970161 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7f531005ed10 con 0x7f53200c4ac0 2024-09-21T02:09:29.908 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:29.908 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:29.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5304073200 msgr2=0x7f5304075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:29.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5304073200 0x7f5304075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f531c0af000 tx=0x7f531c01b330 comp rx=0 tx=0).stop 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 msgr2=0x7f53200c3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 0x7f53200c3e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53100058d0 tx=0x7f531000dde0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 shutdown_connections 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5304073200 0x7f5304075710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f53200cdbf0 0x7f5320156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53200c4ac0 0x7f53200c3e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 --2- 172.21.15.96:0/1713970161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f532006e2c0 0x7f53201504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 >> 172.21.15.96:0/1713970161 conn(0x7f532006eea0 msgr2=0x7f532015ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 shutdown_connections 2024-09-21T02:09:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:29.904+0000 7f532838b640 1 -- 172.21.15.96:0/1713970161 wait complete. 2024-09-21T02:09:29.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:30.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 -- 172.21.15.96:0/1260970781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac06e2c0 msgr2=0x7f9bac0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 --2- 172.21.15.96:0/1260970781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac06e2c0 0x7f9bac0c4580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009900 tx=0x7f9ba801c850 comp rx=0 tx=0).stop 2024-09-21T02:09:30.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 -- 172.21.15.96:0/1260970781 shutdown_connections 2024-09-21T02:09:30.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 --2- 172.21.15.96:0/1260970781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bac0c4ac0 0x7f9bac0c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 --2- 172.21.15.96:0/1260970781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac06e2c0 0x7f9bac0c4580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 -- 172.21.15.96:0/1260970781 >> 172.21.15.96:0/1260970781 conn(0x7f9bac06eea0 msgr2=0x7f9bac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:30.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 -- 172.21.15.96:0/1260970781 shutdown_connections 2024-09-21T02:09:30.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.064+0000 7f9bb3133640 1 -- 172.21.15.96:0/1260970781 wait complete. 2024-09-21T02:09:30.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 Processor -- start 2024-09-21T02:09:30.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 -- start start 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bac06e2c0 0x7f9bac15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac0c4ac0 0x7f9bac158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 0x7f9bac158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0cf2b0 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0cf130 con 0x7f9bac0c4ac0 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb3133640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac0cf430 con 0x7f9bac06e2c0 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb16d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 0x7f9bac158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb16d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 0x7f9bac158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59646/0 (socket says 172.21.15.96:59646) 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb16d0640 1 -- 172.21.15.96:0/3746966065 learned_addr learned my addr 172.21.15.96:0/3746966065 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9ba3fff640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac0c4ac0 0x7f9bac158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.068+0000 7f9bb0ecf640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bac06e2c0 0x7f9bac15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb16d0640 1 -- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bac06e2c0 msgr2=0x7f9bac15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb16d0640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bac06e2c0 0x7f9bac15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb16d0640 1 -- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac0c4ac0 msgr2=0x7f9bac158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb16d0640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac0c4ac0 0x7f9bac158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb16d0640 1 -- 172.21.15.96:0/3746966065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac149550 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb16d0640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 0x7f9bac158ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400c740 tx=0x7f9ba400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4003b80 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac157ea0 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ba400ce50 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400a850 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac158460 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac157c90 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ba400a9f0 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9ba17fa640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9b98073230 0x7f9b98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9ba4094290 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.072+0000 7f9bb0ecf640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9b98073230 0x7f9b98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.076+0000 7f9bb0ecf640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9b98073230 0x7f9b98075740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80af000 tx=0x7f9ba801b090 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:30.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.076+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9ba405e8a0 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.236+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bac0c4580 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.236+0000 7f9ba17fa640 1 -- 172.21.15.96:0/3746966065 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7f9ba40632d0 con 0x7f9bac0cdbf0 2024-09-21T02:09:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:30.243 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9b98073230 msgr2=0x7f9b98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9b98073230 0x7f9b98075740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80af000 tx=0x7f9ba801b090 comp rx=0 tx=0).stop 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 msgr2=0x7f9bac158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 0x7f9bac158ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400c740 tx=0x7f9ba400cc00 comp rx=0 tx=0).stop 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 shutdown_connections 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9b98073230 0x7f9b98075740 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bac0cdbf0 0x7f9bac158ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bac0c4ac0 0x7f9bac158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 --2- 172.21.15.96:0/3746966065 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bac06e2c0 0x7f9bac15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 >> 172.21.15.96:0/3746966065 conn(0x7f9bac06eea0 msgr2=0x7f9bac0b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:30.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 shutdown_connections 2024-09-21T02:09:30.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.240+0000 7f9bb3133640 1 -- 172.21.15.96:0/3746966065 wait complete. 2024-09-21T02:09:30.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:09:30.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 -- 172.21.15.96:0/2462844419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c06e5b0 msgr2=0x7f826c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 --2- 172.21.15.96:0/2462844419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c06e5b0 0x7f826c15bea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8260009900 tx=0x7f826001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 -- 172.21.15.96:0/2462844419 shutdown_connections 2024-09-21T02:09:30.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 --2- 172.21.15.96:0/2462844419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f826c15c3e0 0x7f826c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 --2- 172.21.15.96:0/2462844419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c06e5b0 0x7f826c15bea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 -- 172.21.15.96:0/2462844419 >> 172.21.15.96:0/2462844419 conn(0x7f826c06eea0 msgr2=0x7f826c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:30.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 -- 172.21.15.96:0/2462844419 shutdown_connections 2024-09-21T02:09:30.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.416+0000 7f8270924640 1 -- 172.21.15.96:0/2462844419 wait complete. 2024-09-21T02:09:30.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 Processor -- start 2024-09-21T02:09:30.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 -- start start 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c0ced00 0x7f826c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 0x7f826c14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f826c0b0550 0x7f826c14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1687f0 con 0x7f826c15c3e0 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f826c168670 con 0x7f826c0ced00 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f826c168970 con 0x7f826c0b0550 2024-09-21T02:09:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 0x7f826c14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 0x7f826c14edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59660/0 (socket says 172.21.15.96:59660) 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 -- 172.21.15.96:0/2588955543 learned_addr learned my addr 172.21.15.96:0/2588955543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 -- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f826c0b0550 msgr2=0x7f826c14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f826c0b0550 0x7f826c14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f826a59c640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c0ced00 0x7f826c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 -- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c0ced00 msgr2=0x7f826c14bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c0ced00 0x7f826c14bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 -- 172.21.15.96:0/2588955543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0b0f10 con 0x7f826c15c3e0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8269d9b640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 0x7f826c14edc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82600b0040 tx=0x7f82600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82600b2050 con 0x7f826c15c3e0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82600b3040 con 0x7f826c15c3e0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c14e2e0 con 0x7f826c15c3e0 2024-09-21T02:09:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260003600 con 0x7f826c15c3e0 2024-09-21T02:09:30.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.420+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c14e8a0 con 0x7f826c15c3e0 2024-09-21T02:09:30.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.424+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c15bea0 con 0x7f826c15c3e0 2024-09-21T02:09:30.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.424+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f82600c0020 con 0x7f826c15c3e0 2024-09-21T02:09:30.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.424+0000 7f825affd640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f824c073230 0x7f824c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:30.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.424+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f82601313e0 con 0x7f826c15c3e0 2024-09-21T02:09:30.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.424+0000 7f826a59c640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f824c073230 0x7f824c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:30.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.424+0000 7f826a59c640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f824c073230 0x7f824c075740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f825c000f70 tx=0x7f825c009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:30.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.428+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f82600c7020 con 0x7f826c15c3e0 2024-09-21T02:09:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:30.632+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f826c14e090 con 0x7f826c15c3e0 2024-09-21T02:09:31.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.576+0000 7f825affd640 1 -- 172.21.15.96:0/2588955543 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f82600164d0 con 0x7f826c15c3e0 2024-09-21T02:09:31.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f824c073230 msgr2=0x7f824c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f824c073230 0x7f824c075740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f825c000f70 tx=0x7f825c009040 comp rx=0 tx=0).stop 2024-09-21T02:09:31.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 msgr2=0x7f826c14edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 0x7f826c14edc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82600b0040 tx=0x7f82600040c0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 shutdown_connections 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f824c073230 0x7f824c075740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f826c0b0550 0x7f826c14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f826c15c3e0 0x7f826c14edc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 --2- 172.21.15.96:0/2588955543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f826c0ced00 0x7f826c14bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 >> 172.21.15.96:0/2588955543 conn(0x7f826c06eea0 msgr2=0x7f826c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 shutdown_connections 2024-09-21T02:09:31.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.580+0000 7f8270924640 1 -- 172.21.15.96:0/2588955543 wait complete. 2024-09-21T02:09:31.608 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-21T02:09:31.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:09:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 -- 172.21.15.96:0/370019763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c406e5b0 msgr2=0x7ff6c415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/370019763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c406e5b0 0x7ff6c415bea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009840 tx=0x7ff6ac01c820 comp rx=0 tx=0).stop 2024-09-21T02:09:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 -- 172.21.15.96:0/370019763 shutdown_connections 2024-09-21T02:09:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/370019763 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c415c3e0 0x7ff6c40cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/370019763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c406e5b0 0x7ff6c415bea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 -- 172.21.15.96:0/370019763 >> 172.21.15.96:0/370019763 conn(0x7ff6c406eea0 msgr2=0x7ff6c406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 -- 172.21.15.96:0/370019763 shutdown_connections 2024-09-21T02:09:31.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.764+0000 7ff6c93b4640 1 -- 172.21.15.96:0/370019763 wait complete. 2024-09-21T02:09:31.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 Processor -- start 2024-09-21T02:09:31.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 -- start start 2024-09-21T02:09:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 0x7ff6c40cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c415c3e0 0x7ff6c414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c40ce300 0x7ff6c414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4168840 con 0x7ff6c40ced00 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41686c0 con 0x7ff6c415c3e0 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c41689c0 con 0x7ff6c40ce300 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 0x7ff6c40cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 0x7ff6c40cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59672/0 (socket says 172.21.15.96:59672) 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 -- 172.21.15.96:0/3188024336 learned_addr learned my addr 172.21.15.96:0/3188024336 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c37fe640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c40ce300 0x7ff6c414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6bbfff640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c415c3e0 0x7ff6c414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 -- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c40ce300 msgr2=0x7ff6c414d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c40ce300 0x7ff6c414d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 -- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c415c3e0 msgr2=0x7ff6c414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c415c3e0 0x7ff6c414cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 -- 172.21.15.96:0/3188024336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c4150fa0 con 0x7ff6c40ced00 2024-09-21T02:09:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c37fe640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c40ce300 0x7ff6c414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c2ffd640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 0x7ff6c40cddc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c430 tx=0x7ff6b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4011790 con 0x7ff6c40ced00 2024-09-21T02:09:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6b4017070 con 0x7ff6c40ced00 2024-09-21T02:09:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4011cc0 con 0x7ff6c40ced00 2024-09-21T02:09:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c4151240 con 0x7ff6c40ced00 2024-09-21T02:09:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.768+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41475c0 con 0x7ff6c40ced00 2024-09-21T02:09:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.772+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c415bea0 con 0x7ff6c40ced00 2024-09-21T02:09:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.772+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6b401c460 con 0x7ff6c40ced00 2024-09-21T02:09:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.772+0000 7ff6bb7fe640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff6a8073230 0x7ff6a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.772+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff6b40943d0 con 0x7ff6c40ced00 2024-09-21T02:09:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.772+0000 7ff6bbfff640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff6a8073230 0x7ff6a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.772+0000 7ff6bbfff640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff6a8073230 0x7ff6a8075740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0af000 tx=0x7ff6ac01b060 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.776+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff6c4150fa0 con 0x7ff6c40ced00 2024-09-21T02:09:31.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:31.972+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff6c4151410 con 0x7ff6c40ced00 2024-09-21T02:09:32.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:09:32.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:09:32.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.592+0000 7ff6bb7fe640 1 -- 172.21.15.96:0/3188024336 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7ff6b405f010 con 0x7ff6c40ced00 2024-09-21T02:09:32.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff6a8073230 msgr2=0x7ff6a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff6a8073230 0x7ff6a8075740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0af000 tx=0x7ff6ac01b060 comp rx=0 tx=0).stop 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 msgr2=0x7ff6c40cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 0x7ff6c40cddc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c430 tx=0x7ff6b400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 shutdown_connections 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff6a8073230 0x7ff6a8075740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6c40ce300 0x7ff6c414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c415c3e0 0x7ff6c414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 --2- 172.21.15.96:0/3188024336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6c40ced00 0x7ff6c40cddc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 >> 172.21.15.96:0/3188024336 conn(0x7ff6c406eea0 msgr2=0x7ff6c415d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 shutdown_connections 2024-09-21T02:09:32.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.596+0000 7ff6c93b4640 1 -- 172.21.15.96:0/3188024336 wait complete. 2024-09-21T02:09:32.623 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:09:32.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:32.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f501dabf640 1 -- 172.21.15.96:0/2037609686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5004003fe0 msgr2=0x7f5004004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:32.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f501dabf640 1 --2- 172.21.15.96:0/2037609686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5004003fe0 0x7f5004004810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5008009eb0 tx=0x7f500801d5c0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f5017fff640 1 -- 172.21.15.96:0/2037609686 reap_dead start 2024-09-21T02:09:32.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f501dabf640 1 -- 172.21.15.96:0/2037609686 shutdown_connections 2024-09-21T02:09:32.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f501dabf640 1 -- 172.21.15.96:0/2037609686 >> 172.21.15.96:0/2037609686 conn(0x7f50180b42f0 msgr2=0x7f50180d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:32.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f501dabf640 1 -- 172.21.15.96:0/2037609686 shutdown_connections 2024-09-21T02:09:32.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.776+0000 7f501dabf640 1 -- 172.21.15.96:0/2037609686 wait complete. 2024-09-21T02:09:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 Processor -- start 2024-09-21T02:09:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 -- start start 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004003fe0 0x7f50181f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 0x7f50180c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50181555a0 0x7f50181fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5018167d90 con 0x7f50180c80b0 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5018167c10 con 0x7f5004003fe0 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5018167f10 con 0x7f50181555a0 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 0x7f50180c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 0x7f50180c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59688/0 (socket says 172.21.15.96:59688) 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 -- 172.21.15.96:0/3788415752 learned_addr learned my addr 172.21.15.96:0/3788415752 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f50177fe640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004003fe0 0x7f50181f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5017fff640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50181555a0 0x7f50181fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 -- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50181555a0 msgr2=0x7f50181fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50181555a0 0x7f50181fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 -- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004003fe0 msgr2=0x7f50181f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004003fe0 0x7f50181f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 -- 172.21.15.96:0/3788415752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50180c85a0 con 0x7f50180c80b0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5017fff640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50181555a0 0x7f50181fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f5016ffd640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 0x7f50180c6490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5008009e00 tx=0x7f50080a9550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008004ad0 con 0x7f50180c80b0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5008004c70 con 0x7f50180c80b0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50180c8880 con 0x7f50180c80b0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008004e60 con 0x7f50180c80b0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.780+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181fc500 con 0x7f50180c80b0 2024-09-21T02:09:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.784+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50180d1780 con 0x7f50180c80b0 2024-09-21T02:09:32.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.784+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50080b9030 con 0x7f50180c80b0 2024-09-21T02:09:32.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.784+0000 7f500ffff640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4ff8073200 0x7f4ff8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:32.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.784+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f500812f070 con 0x7f50180c80b0 2024-09-21T02:09:32.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.784+0000 7f50177fe640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4ff8073200 0x7f4ff8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:32.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.784+0000 7f50177fe640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4ff8073200 0x7f4ff8075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5000009a10 tx=0x7f50000091e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:32.788+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50080ad030 con 0x7f50180c80b0 2024-09-21T02:09:33.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.000+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5018166840 con 0x7f50180c80b0 2024-09-21T02:09:33.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f500ffff640 1 -- 172.21.15.96:0/3788415752 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f5008017300 con 0x7f50180c80b0 2024-09-21T02:09:33.007 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:33.007 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:09:33.007 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:09:33.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4ff8073200 msgr2=0x7f4ff8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4ff8073200 0x7f4ff8075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5000009a10 tx=0x7f50000091e0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 msgr2=0x7f50180c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 0x7f50180c6490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5008009e00 tx=0x7f50080a9550 comp rx=0 tx=0).stop 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 shutdown_connections 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4ff8073200 0x7f4ff8075710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50181555a0 0x7f50181fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50180c80b0 0x7f50180c6490 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 --2- 172.21.15.96:0/3788415752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004003fe0 0x7f50181f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 >> 172.21.15.96:0/3788415752 conn(0x7f50180b42f0 msgr2=0x7f50180cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 shutdown_connections 2024-09-21T02:09:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.004+0000 7f501dabf640 1 -- 172.21.15.96:0/3788415752 wait complete. 2024-09-21T02:09:33.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:33.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 -- 172.21.15.96:0/1855613864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef006dea0 msgr2=0x7efef006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 --2- 172.21.15.96:0/1855613864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef006dea0 0x7efef006e2b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efef00cdb80 tx=0x7efee001c6b0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 -- 172.21.15.96:0/1855613864 shutdown_connections 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 --2- 172.21.15.96:0/1855613864 >> v2:172.21.15.122:3300/0 conn(0x7efef00cab60 0x7efef00caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 --2- 172.21.15.96:0/1855613864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef006dea0 0x7efef006e2b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 -- 172.21.15.96:0/1855613864 >> 172.21.15.96:0/1855613864 conn(0x7efef006eea0 msgr2=0x7efef006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 -- 172.21.15.96:0/1855613864 shutdown_connections 2024-09-21T02:09:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.180+0000 7efef704c640 1 -- 172.21.15.96:0/1855613864 wait complete. 2024-09-21T02:09:33.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 Processor -- start 2024-09-21T02:09:33.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- start start 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef006dea0 0x7efef014e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef00cab60 0x7efef00cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 0x7efef00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efef00cbd00 con 0x7efef0155240 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efef00cbb80 con 0x7efef006dea0 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efef00cbe80 con 0x7efef00cab60 2024-09-21T02:09:33.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 0x7efef00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 0x7efef00cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59702/0 (socket says 172.21.15.96:59702) 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 -- 172.21.15.96:0/2289389213 learned_addr learned my addr 172.21.15.96:0/2289389213 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efeeffff640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef00cab60 0x7efef00cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef4de8640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef006dea0 0x7efef014e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 -- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef00cab60 msgr2=0x7efef00cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef00cab60 0x7efef00cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 -- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef006dea0 msgr2=0x7efef014e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef006dea0 0x7efef014e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 -- 172.21.15.96:0/2289389213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef014e9d0 con 0x7efef0155240 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efeeffff640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef00cab60 0x7efef00cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef55e9640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 0x7efef00cefc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efee800ca60 tx=0x7efee800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8007d90 con 0x7efef0155240 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee8004c20 con 0x7efef0155240 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef00c67b0 con 0x7efef0155240 2024-09-21T02:09:33.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8005280 con 0x7efef0155240 2024-09-21T02:09:33.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef014f880 con 0x7efef0155240 2024-09-21T02:09:33.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.184+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef006e2b0 con 0x7efef0155240 2024-09-21T02:09:33.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.188+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efee8005420 con 0x7efef0155240 2024-09-21T02:09:33.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.188+0000 7efeed7fa640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efedc073230 0x7efedc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.188+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efee8095120 con 0x7efef0155240 2024-09-21T02:09:33.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.188+0000 7efef4de8640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efedc073230 0x7efedc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.188+0000 7efef4de8640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efedc073230 0x7efedc075740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efee400b810 tx=0x7efee4008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:33.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.192+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efee8014030 con 0x7efef0155240 2024-09-21T02:09:33.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.348+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efef006e9d0 con 0x7efef0155240 2024-09-21T02:09:33.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.352+0000 7efeed7fa640 1 -- 172.21.15.96:0/2289389213 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7efee805fe80 con 0x7efef0155240 2024-09-21T02:09:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:33.357 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:33.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efedc073230 msgr2=0x7efedc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efedc073230 0x7efedc075740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efee400b810 tx=0x7efee4008040 comp rx=0 tx=0).stop 2024-09-21T02:09:33.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 msgr2=0x7efef00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 0x7efef00cefc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efee800ca60 tx=0x7efee800cf20 comp rx=0 tx=0).stop 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 shutdown_connections 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efedc073230 0x7efedc075740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef0155240 0x7efef00cefc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef00cab60 0x7efef00cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 --2- 172.21.15.96:0/2289389213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef006dea0 0x7efef014e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 >> 172.21.15.96:0/2289389213 conn(0x7efef006eea0 msgr2=0x7efef006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 shutdown_connections 2024-09-21T02:09:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.356+0000 7efef704c640 1 -- 172.21.15.96:0/2289389213 wait complete. 2024-09-21T02:09:33.379 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:09:33.379 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:09:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 -- 172.21.15.96:0/3242068649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 msgr2=0x7fa4ec06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 --2- 172.21.15.96:0/3242068649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec06e3b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec15cad0 tx=0x7fa4e001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 -- 172.21.15.96:0/3242068649 shutdown_connections 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 --2- 172.21.15.96:0/3242068649 >> v2:172.21.15.122:3300/0 conn(0x7fa4ec0d00e0 0x7fa4ec0d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 --2- 172.21.15.96:0/3242068649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec06e3b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 -- 172.21.15.96:0/3242068649 >> 172.21.15.96:0/3242068649 conn(0x7fa4ec0b42f0 msgr2=0x7fa4ec0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 -- 172.21.15.96:0/3242068649 shutdown_connections 2024-09-21T02:09:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.520+0000 7fa4f3292640 1 -- 172.21.15.96:0/3242068649 wait complete. 2024-09-21T02:09:33.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 Processor -- start 2024-09-21T02:09:33.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 -- start start 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa4ec0d00e0 0x7fa4ec1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec1624f0 0x7fa4ec152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec16c7c0 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec16c640 con 0x7fa4ec1624f0 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec16c940 con 0x7fa4ec0d00e0 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59706/0 (socket says 172.21.15.96:59706) 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 -- 172.21.15.96:0/1705966478 learned_addr learned my addr 172.21.15.96:0/1705966478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f182f640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec1624f0 0x7fa4ec152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f082d640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa4ec0d00e0 0x7fa4ec1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 -- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa4ec0d00e0 msgr2=0x7fa4ec1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa4ec0d00e0 0x7fa4ec1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 -- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec1624f0 msgr2=0x7fa4ec152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec1624f0 0x7fa4ec152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 -- 172.21.15.96:0/1705966478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0b6b10 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f182f640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec1624f0 0x7fa4ec152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f102e640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec0ccb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00b0040 tx=0x7fa4e00a5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b2040 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e00041b0 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.524+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec0b6df0 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003600 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec153a40 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec06e3b0 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4e00bf020 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4ddffb640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa4d0073230 0x7fa4d0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa4e012e850 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4f082d640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa4d0073230 0x7fa4d0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.528+0000 7fa4f082d640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa4d0073230 0x7fa4d0075740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec159330 tx=0x7fa4e8005f70 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.532+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa4e00c6020 con 0x7fa4ec06dfa0 2024-09-21T02:09:33.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.692+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4ec152ea0 con 0x7fa4d0073230 2024-09-21T02:09:33.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.708+0000 7fa4ddffb640 1 -- 172.21.15.96:0/1705966478 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7fa4ec152ea0 con 0x7fa4d0073230 2024-09-21T02:09:33.715 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:33.719 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:09:33.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa4d0073230 msgr2=0x7fa4d0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa4d0073230 0x7fa4d0075740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec159330 tx=0x7fa4e8005f70 comp rx=0 tx=0).stop 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 msgr2=0x7fa4ec0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec0ccb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00b0040 tx=0x7fa4e00a5580 comp rx=0 tx=0).stop 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 shutdown_connections 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa4d0073230 0x7fa4d0075740 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec1624f0 0x7fa4ec152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa4ec0d00e0 0x7fa4ec1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 --2- 172.21.15.96:0/1705966478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4ec06dfa0 0x7fa4ec0ccb50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.712+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 >> 172.21.15.96:0/1705966478 conn(0x7fa4ec0b42f0 msgr2=0x7fa4ec0cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.716+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 shutdown_connections 2024-09-21T02:09:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.716+0000 7fa4f3292640 1 -- 172.21.15.96:0/1705966478 wait complete. 2024-09-21T02:09:33.736 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T02:09:32.505056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223768,"kb_used_data":3368,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750952,"statfs":{"total":773094113280,"available":772864974848,"internally_reserved":0,"allocated":3448832,"data_stored":2054672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005923"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723570+0000","last_change":"2024-09-21T02:09:10.527965+0000","last_active":"2024-09-21T02:09:16.723570+0000","last_peered":"2024-09-21T02:09:16.723570+0000","last_clean":"2024-09-21T02:09:16.723570+0000","last_became_active":"2024-09-21T02:09:10.527508+0000","last_became_peered":"2024-09-21T02:09:10.527508+0000","last_unstale":"2024-09-21T02:09:16.723570+0000","last_undegraded":"2024-09-21T02:09:16.723570+0000","last_fullsized":"2024-09-21T02:09:16.723570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:35.986615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590616+0000","last_change":"2024-09-21T02:09:11.546557+0000","last_active":"2024-09-21T02:09:16.590616+0000","last_peered":"2024-09-21T02:09:16.590616+0000","last_clean":"2024-09-21T02:09:16.590616+0000","last_became_active":"2024-09-21T02:09:11.546158+0000","last_became_peered":"2024-09-21T02:09:11.546158+0000","last_unstale":"2024-09-21T02:09:16.590616+0000","last_undegraded":"2024-09-21T02:09:16.590616+0000","last_fullsized":"2024-09-21T02:09:16.590616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588876+0000","last_change":"2024-09-21T02:09:15.612941+0000","last_active":"2024-09-21T02:09:16.588876+0000","last_peered":"2024-09-21T02:09:16.588876+0000","last_clean":"2024-09-21T02:09:16.588876+0000","last_became_active":"2024-09-21T02:09:15.612114+0000","last_became_peered":"2024-09-21T02:09:15.612114+0000","last_unstale":"2024-09-21T02:09:16.588876+0000","last_undegraded":"2024-09-21T02:09:16.588876+0000","last_fullsized":"2024-09-21T02:09:16.588876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590737+0000","last_change":"2024-09-21T02:09:10.523298+0000","last_active":"2024-09-21T02:09:16.590737+0000","last_peered":"2024-09-21T02:09:16.590737+0000","last_clean":"2024-09-21T02:09:16.590737+0000","last_became_active":"2024-09-21T02:09:10.521768+0000","last_became_peered":"2024-09-21T02:09:10.521768+0000","last_unstale":"2024-09-21T02:09:16.590737+0000","last_undegraded":"2024-09-21T02:09:16.590737+0000","last_fullsized":"2024-09-21T02:09:16.590737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723214+0000","last_change":"2024-09-21T02:09:11.573485+0000","last_active":"2024-09-21T02:09:16.723214+0000","last_peered":"2024-09-21T02:09:16.723214+0000","last_clean":"2024-09-21T02:09:16.723214+0000","last_became_active":"2024-09-21T02:09:11.572854+0000","last_became_peered":"2024-09-21T02:09:11.572854+0000","last_unstale":"2024-09-21T02:09:16.723214+0000","last_undegraded":"2024-09-21T02:09:16.723214+0000","last_fullsized":"2024-09-21T02:09:16.723214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591993+0000","last_change":"2024-09-21T02:09:15.629971+0000","last_active":"2024-09-21T02:09:16.591993+0000","last_peered":"2024-09-21T02:09:16.591993+0000","last_clean":"2024-09-21T02:09:16.591993+0000","last_became_active":"2024-09-21T02:09:15.628916+0000","last_became_peered":"2024-09-21T02:09:15.628916+0000","last_unstale":"2024-09-21T02:09:16.591993+0000","last_undegraded":"2024-09-21T02:09:16.591993+0000","last_fullsized":"2024-09-21T02:09:16.591993+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:16.541985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592632+0000","last_change":"2024-09-21T02:09:10.549868+0000","last_active":"2024-09-21T02:09:16.592632+0000","last_peered":"2024-09-21T02:09:16.592632+0000","last_clean":"2024-09-21T02:09:16.592632+0000","last_became_active":"2024-09-21T02:09:10.549154+0000","last_became_peered":"2024-09-21T02:09:10.549154+0000","last_unstale":"2024-09-21T02:09:16.592632+0000","last_undegraded":"2024-09-21T02:09:16.592632+0000","last_fullsized":"2024-09-21T02:09:16.592632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:42.470809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723996+0000","last_change":"2024-09-21T02:09:11.556559+0000","last_active":"2024-09-21T02:09:16.723996+0000","last_peered":"2024-09-21T02:09:16.723996+0000","last_clean":"2024-09-21T02:09:16.723996+0000","last_became_active":"2024-09-21T02:09:11.555720+0000","last_became_peered":"2024-09-21T02:09:11.555720+0000","last_unstale":"2024-09-21T02:09:16.723996+0000","last_undegraded":"2024-09-21T02:09:16.723996+0000","last_fullsized":"2024-09-21T02:09:16.723996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:43.740023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596031+0000","last_change":"2024-09-21T02:09:15.626432+0000","last_active":"2024-09-21T02:09:16.596031+0000","last_peered":"2024-09-21T02:09:16.596031+0000","last_clean":"2024-09-21T02:09:16.596031+0000","last_became_active":"2024-09-21T02:09:15.620746+0000","last_became_peered":"2024-09-21T02:09:15.620746+0000","last_unstale":"2024-09-21T02:09:16.596031+0000","last_undegraded":"2024-09-21T02:09:16.596031+0000","last_fullsized":"2024-09-21T02:09:16.596031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:46.776489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591909+0000","last_change":"2024-09-21T02:09:10.552140+0000","last_active":"2024-09-21T02:09:16.591909+0000","last_peered":"2024-09-21T02:09:16.591909+0000","last_clean":"2024-09-21T02:09:16.591909+0000","last_became_active":"2024-09-21T02:09:10.551607+0000","last_became_peered":"2024-09-21T02:09:10.551607+0000","last_unstale":"2024-09-21T02:09:16.591909+0000","last_undegraded":"2024-09-21T02:09:16.591909+0000","last_fullsized":"2024-09-21T02:09:16.591909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724056+0000","last_change":"2024-09-21T02:09:11.557870+0000","last_active":"2024-09-21T02:09:16.724056+0000","last_peered":"2024-09-21T02:09:16.724056+0000","last_clean":"2024-09-21T02:09:16.724056+0000","last_became_active":"2024-09-21T02:09:11.556258+0000","last_became_peered":"2024-09-21T02:09:11.556258+0000","last_unstale":"2024-09-21T02:09:16.724056+0000","last_undegraded":"2024-09-21T02:09:16.724056+0000","last_fullsized":"2024-09-21T02:09:16.724056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592325+0000","last_change":"2024-09-21T02:09:15.639499+0000","last_active":"2024-09-21T02:09:16.592325+0000","last_peered":"2024-09-21T02:09:16.592325+0000","last_clean":"2024-09-21T02:09:16.592325+0000","last_became_active":"2024-09-21T02:09:15.638499+0000","last_became_peered":"2024-09-21T02:09:15.638499+0000","last_unstale":"2024-09-21T02:09:16.592325+0000","last_undegraded":"2024-09-21T02:09:16.592325+0000","last_fullsized":"2024-09-21T02:09:16.592325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:22.032011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722630+0000","last_change":"2024-09-21T02:09:11.561488+0000","last_active":"2024-09-21T02:09:16.722630+0000","last_peered":"2024-09-21T02:09:16.722630+0000","last_clean":"2024-09-21T02:09:16.722630+0000","last_became_active":"2024-09-21T02:09:11.560683+0000","last_became_peered":"2024-09-21T02:09:11.560683+0000","last_unstale":"2024-09-21T02:09:16.722630+0000","last_undegraded":"2024-09-21T02:09:16.722630+0000","last_fullsized":"2024-09-21T02:09:16.722630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592616+0000","last_change":"2024-09-21T02:09:10.552547+0000","last_active":"2024-09-21T02:09:16.592616+0000","last_peered":"2024-09-21T02:09:16.592616+0000","last_clean":"2024-09-21T02:09:16.592616+0000","last_became_active":"2024-09-21T02:09:10.552056+0000","last_became_peered":"2024-09-21T02:09:10.552056+0000","last_unstale":"2024-09-21T02:09:16.592616+0000","last_undegraded":"2024-09-21T02:09:16.592616+0000","last_fullsized":"2024-09-21T02:09:16.592616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594025+0000","last_change":"2024-09-21T02:09:15.611360+0000","last_active":"2024-09-21T02:09:16.594025+0000","last_peered":"2024-09-21T02:09:16.594025+0000","last_clean":"2024-09-21T02:09:16.594025+0000","last_became_active":"2024-09-21T02:09:15.609991+0000","last_became_peered":"2024-09-21T02:09:15.609991+0000","last_unstale":"2024-09-21T02:09:16.594025+0000","last_undegraded":"2024-09-21T02:09:16.594025+0000","last_fullsized":"2024-09-21T02:09:16.594025+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:24.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589470+0000","last_change":"2024-09-21T02:09:11.545847+0000","last_active":"2024-09-21T02:09:16.589470+0000","last_peered":"2024-09-21T02:09:16.589470+0000","last_clean":"2024-09-21T02:09:16.589470+0000","last_became_active":"2024-09-21T02:09:11.543606+0000","last_became_peered":"2024-09-21T02:09:11.543606+0000","last_unstale":"2024-09-21T02:09:16.589470+0000","last_undegraded":"2024-09-21T02:09:16.589470+0000","last_fullsized":"2024-09-21T02:09:16.589470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720543+0000","last_change":"2024-09-21T02:09:10.524519+0000","last_active":"2024-09-21T02:09:16.720543+0000","last_peered":"2024-09-21T02:09:16.720543+0000","last_clean":"2024-09-21T02:09:16.720543+0000","last_became_active":"2024-09-21T02:09:10.524074+0000","last_became_peered":"2024-09-21T02:09:10.524074+0000","last_unstale":"2024-09-21T02:09:16.720543+0000","last_undegraded":"2024-09-21T02:09:16.720543+0000","last_fullsized":"2024-09-21T02:09:16.720543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594281+0000","last_change":"2024-09-21T02:09:15.618974+0000","last_active":"2024-09-21T02:09:16.594281+0000","last_peered":"2024-09-21T02:09:16.594281+0000","last_clean":"2024-09-21T02:09:16.594281+0000","last_became_active":"2024-09-21T02:09:15.617774+0000","last_became_peered":"2024-09-21T02:09:15.617774+0000","last_unstale":"2024-09-21T02:09:16.594281+0000","last_undegraded":"2024-09-21T02:09:16.594281+0000","last_fullsized":"2024-09-21T02:09:16.594281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:47.809682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589087+0000","last_change":"2024-09-21T02:09:11.538383+0000","last_active":"2024-09-21T02:09:16.589087+0000","last_peered":"2024-09-21T02:09:16.589087+0000","last_clean":"2024-09-21T02:09:16.589087+0000","last_became_active":"2024-09-21T02:09:11.537569+0000","last_became_peered":"2024-09-21T02:09:11.537569+0000","last_unstale":"2024-09-21T02:09:16.589087+0000","last_undegraded":"2024-09-21T02:09:16.589087+0000","last_fullsized":"2024-09-21T02:09:16.589087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720139+0000","last_change":"2024-09-21T02:09:10.537296+0000","last_active":"2024-09-21T02:09:16.720139+0000","last_peered":"2024-09-21T02:09:16.720139+0000","last_clean":"2024-09-21T02:09:16.720139+0000","last_became_active":"2024-09-21T02:09:10.536913+0000","last_became_peered":"2024-09-21T02:09:10.536913+0000","last_unstale":"2024-09-21T02:09:16.720139+0000","last_undegraded":"2024-09-21T02:09:16.720139+0000","last_fullsized":"2024-09-21T02:09:16.720139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720276+0000","last_change":"2024-09-21T02:09:15.606253+0000","last_active":"2024-09-21T02:09:16.720276+0000","last_peered":"2024-09-21T02:09:16.720276+0000","last_clean":"2024-09-21T02:09:16.720276+0000","last_became_active":"2024-09-21T02:09:15.605847+0000","last_became_peered":"2024-09-21T02:09:15.605847+0000","last_unstale":"2024-09-21T02:09:16.720276+0000","last_undegraded":"2024-09-21T02:09:16.720276+0000","last_fullsized":"2024-09-21T02:09:16.720276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594287+0000","last_change":"2024-09-21T02:09:11.541952+0000","last_active":"2024-09-21T02:09:16.594287+0000","last_peered":"2024-09-21T02:09:16.594287+0000","last_clean":"2024-09-21T02:09:16.594287+0000","last_became_active":"2024-09-21T02:09:11.540560+0000","last_became_peered":"2024-09-21T02:09:11.540560+0000","last_unstale":"2024-09-21T02:09:16.594287+0000","last_undegraded":"2024-09-21T02:09:16.594287+0000","last_fullsized":"2024-09-21T02:09:16.594287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:25.306228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591648+0000","last_change":"2024-09-21T02:09:10.544405+0000","last_active":"2024-09-21T02:09:16.591648+0000","last_peered":"2024-09-21T02:09:16.591648+0000","last_clean":"2024-09-21T02:09:16.591648+0000","last_became_active":"2024-09-21T02:09:10.540771+0000","last_became_peered":"2024-09-21T02:09:10.540771+0000","last_unstale":"2024-09-21T02:09:16.591648+0000","last_undegraded":"2024-09-21T02:09:16.591648+0000","last_fullsized":"2024-09-21T02:09:16.591648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593729+0000","last_change":"2024-09-21T02:09:15.626770+0000","last_active":"2024-09-21T02:09:16.593729+0000","last_peered":"2024-09-21T02:09:16.593729+0000","last_clean":"2024-09-21T02:09:16.593729+0000","last_became_active":"2024-09-21T02:09:15.623532+0000","last_became_peered":"2024-09-21T02:09:15.623532+0000","last_unstale":"2024-09-21T02:09:16.593729+0000","last_undegraded":"2024-09-21T02:09:16.593729+0000","last_fullsized":"2024-09-21T02:09:16.593729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:52.883467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722937+0000","last_change":"2024-09-21T02:09:11.539618+0000","last_active":"2024-09-21T02:09:16.722937+0000","last_peered":"2024-09-21T02:09:16.722937+0000","last_clean":"2024-09-21T02:09:16.722937+0000","last_became_active":"2024-09-21T02:09:11.538801+0000","last_became_peered":"2024-09-21T02:09:11.538801+0000","last_unstale":"2024-09-21T02:09:16.722937+0000","last_undegraded":"2024-09-21T02:09:16.722937+0000","last_fullsized":"2024-09-21T02:09:16.722937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726772+0000","last_change":"2024-09-21T02:09:10.545874+0000","last_active":"2024-09-21T02:09:16.726772+0000","last_peered":"2024-09-21T02:09:16.726772+0000","last_clean":"2024-09-21T02:09:16.726772+0000","last_became_active":"2024-09-21T02:09:10.545200+0000","last_became_peered":"2024-09-21T02:09:10.545200+0000","last_unstale":"2024-09-21T02:09:16.726772+0000","last_undegraded":"2024-09-21T02:09:16.726772+0000","last_fullsized":"2024-09-21T02:09:16.726772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593400+0000","last_change":"2024-09-21T02:09:15.630472+0000","last_active":"2024-09-21T02:09:16.593400+0000","last_peered":"2024-09-21T02:09:16.593400+0000","last_clean":"2024-09-21T02:09:16.593400+0000","last_became_active":"2024-09-21T02:09:15.628764+0000","last_became_peered":"2024-09-21T02:09:15.628764+0000","last_unstale":"2024-09-21T02:09:16.593400+0000","last_undegraded":"2024-09-21T02:09:16.593400+0000","last_fullsized":"2024-09-21T02:09:16.593400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:21.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591110+0000","last_change":"2024-09-21T02:09:11.543344+0000","last_active":"2024-09-21T02:09:16.591110+0000","last_peered":"2024-09-21T02:09:16.591110+0000","last_clean":"2024-09-21T02:09:16.591110+0000","last_became_active":"2024-09-21T02:09:11.542450+0000","last_became_peered":"2024-09-21T02:09:11.542450+0000","last_unstale":"2024-09-21T02:09:16.591110+0000","last_undegraded":"2024-09-21T02:09:16.591110+0000","last_fullsized":"2024-09-21T02:09:16.591110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:49.206820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719673+0000","last_change":"2024-09-21T02:09:10.541105+0000","last_active":"2024-09-21T02:09:16.719673+0000","last_peered":"2024-09-21T02:09:16.719673+0000","last_clean":"2024-09-21T02:09:16.719673+0000","last_became_active":"2024-09-21T02:09:10.540744+0000","last_became_peered":"2024-09-21T02:09:10.540744+0000","last_unstale":"2024-09-21T02:09:16.719673+0000","last_undegraded":"2024-09-21T02:09:16.719673+0000","last_fullsized":"2024-09-21T02:09:16.719673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:02.568801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588758+0000","last_change":"2024-09-21T02:09:15.613004+0000","last_active":"2024-09-21T02:09:16.588758+0000","last_peered":"2024-09-21T02:09:16.588758+0000","last_clean":"2024-09-21T02:09:16.588758+0000","last_became_active":"2024-09-21T02:09:15.612265+0000","last_became_peered":"2024-09-21T02:09:15.612265+0000","last_unstale":"2024-09-21T02:09:16.588758+0000","last_undegraded":"2024-09-21T02:09:16.588758+0000","last_fullsized":"2024-09-21T02:09:16.588758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721841+0000","last_change":"2024-09-21T02:09:11.528642+0000","last_active":"2024-09-21T02:09:16.721841+0000","last_peered":"2024-09-21T02:09:16.721841+0000","last_clean":"2024-09-21T02:09:16.721841+0000","last_became_active":"2024-09-21T02:09:11.526472+0000","last_became_peered":"2024-09-21T02:09:11.526472+0000","last_unstale":"2024-09-21T02:09:16.721841+0000","last_undegraded":"2024-09-21T02:09:16.721841+0000","last_fullsized":"2024-09-21T02:09:16.721841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593724+0000","last_change":"2024-09-21T02:09:10.553157+0000","last_active":"2024-09-21T02:09:16.593724+0000","last_peered":"2024-09-21T02:09:16.593724+0000","last_clean":"2024-09-21T02:09:16.593724+0000","last_became_active":"2024-09-21T02:09:10.552668+0000","last_became_peered":"2024-09-21T02:09:10.552668+0000","last_unstale":"2024-09-21T02:09:16.593724+0000","last_undegraded":"2024-09-21T02:09:16.593724+0000","last_fullsized":"2024-09-21T02:09:16.593724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:02.667450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723243+0000","last_change":"2024-09-21T02:09:15.626993+0000","last_active":"2024-09-21T02:09:16.723243+0000","last_peered":"2024-09-21T02:09:16.723243+0000","last_clean":"2024-09-21T02:09:16.723243+0000","last_became_active":"2024-09-21T02:09:15.626367+0000","last_became_peered":"2024-09-21T02:09:15.626367+0000","last_unstale":"2024-09-21T02:09:16.723243+0000","last_undegraded":"2024-09-21T02:09:16.723243+0000","last_fullsized":"2024-09-21T02:09:16.723243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:31.903642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724350+0000","last_change":"2024-09-21T02:09:11.557042+0000","last_active":"2024-09-21T02:09:16.724350+0000","last_peered":"2024-09-21T02:09:16.724350+0000","last_clean":"2024-09-21T02:09:16.724350+0000","last_became_active":"2024-09-21T02:09:11.555201+0000","last_became_peered":"2024-09-21T02:09:11.555201+0000","last_unstale":"2024-09-21T02:09:16.724350+0000","last_undegraded":"2024-09-21T02:09:16.724350+0000","last_fullsized":"2024-09-21T02:09:16.724350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722129+0000","last_change":"2024-09-21T02:09:10.516313+0000","last_active":"2024-09-21T02:09:16.722129+0000","last_peered":"2024-09-21T02:09:16.722129+0000","last_clean":"2024-09-21T02:09:16.722129+0000","last_became_active":"2024-09-21T02:09:10.515863+0000","last_became_peered":"2024-09-21T02:09:10.515863+0000","last_unstale":"2024-09-21T02:09:16.722129+0000","last_undegraded":"2024-09-21T02:09:16.722129+0000","last_fullsized":"2024-09-21T02:09:16.722129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719183+0000","last_change":"2024-09-21T02:09:15.624503+0000","last_active":"2024-09-21T02:09:16.719183+0000","last_peered":"2024-09-21T02:09:16.719183+0000","last_clean":"2024-09-21T02:09:16.719183+0000","last_became_active":"2024-09-21T02:09:15.622674+0000","last_became_peered":"2024-09-21T02:09:15.622674+0000","last_unstale":"2024-09-21T02:09:16.719183+0000","last_undegraded":"2024-09-21T02:09:16.719183+0000","last_fullsized":"2024-09-21T02:09:16.719183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:32.890032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596034+0000","last_change":"2024-09-21T02:09:11.550090+0000","last_active":"2024-09-21T02:09:16.596034+0000","last_peered":"2024-09-21T02:09:16.596034+0000","last_clean":"2024-09-21T02:09:16.596034+0000","last_became_active":"2024-09-21T02:09:11.549246+0000","last_became_peered":"2024-09-21T02:09:11.549246+0000","last_unstale":"2024-09-21T02:09:16.596034+0000","last_undegraded":"2024-09-21T02:09:16.596034+0000","last_fullsized":"2024-09-21T02:09:16.596034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.842699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725146+0000","last_change":"2024-09-21T02:09:10.545849+0000","last_active":"2024-09-21T02:09:16.725146+0000","last_peered":"2024-09-21T02:09:16.725146+0000","last_clean":"2024-09-21T02:09:16.725146+0000","last_became_active":"2024-09-21T02:09:10.544961+0000","last_became_peered":"2024-09-21T02:09:10.544961+0000","last_unstale":"2024-09-21T02:09:16.725146+0000","last_undegraded":"2024-09-21T02:09:16.725146+0000","last_fullsized":"2024-09-21T02:09:16.725146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:43.073819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721335+0000","last_change":"2024-09-21T02:09:15.611136+0000","last_active":"2024-09-21T02:09:16.721335+0000","last_peered":"2024-09-21T02:09:16.721335+0000","last_clean":"2024-09-21T02:09:16.721335+0000","last_became_active":"2024-09-21T02:09:15.610710+0000","last_became_peered":"2024-09-21T02:09:15.610710+0000","last_unstale":"2024-09-21T02:09:16.721335+0000","last_undegraded":"2024-09-21T02:09:16.721335+0000","last_fullsized":"2024-09-21T02:09:16.721335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:11.070735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719495+0000","last_change":"2024-09-21T02:09:11.526818+0000","last_active":"2024-09-21T02:09:16.719495+0000","last_peered":"2024-09-21T02:09:16.719495+0000","last_clean":"2024-09-21T02:09:16.719495+0000","last_became_active":"2024-09-21T02:09:11.525899+0000","last_became_peered":"2024-09-21T02:09:11.525899+0000","last_unstale":"2024-09-21T02:09:16.719495+0000","last_undegraded":"2024-09-21T02:09:16.719495+0000","last_fullsized":"2024-09-21T02:09:16.719495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720218+0000","last_change":"2024-09-21T02:09:10.526158+0000","last_active":"2024-09-21T02:09:16.720218+0000","last_peered":"2024-09-21T02:09:16.720218+0000","last_clean":"2024-09-21T02:09:16.720218+0000","last_became_active":"2024-09-21T02:09:10.525593+0000","last_became_peered":"2024-09-21T02:09:10.525593+0000","last_unstale":"2024-09-21T02:09:16.720218+0000","last_undegraded":"2024-09-21T02:09:16.720218+0000","last_fullsized":"2024-09-21T02:09:16.720218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720063+0000","last_change":"2024-09-21T02:09:15.623852+0000","last_active":"2024-09-21T02:09:16.720063+0000","last_peered":"2024-09-21T02:09:16.720063+0000","last_clean":"2024-09-21T02:09:16.720063+0000","last_became_active":"2024-09-21T02:09:15.623080+0000","last_became_peered":"2024-09-21T02:09:15.623080+0000","last_unstale":"2024-09-21T02:09:16.720063+0000","last_undegraded":"2024-09-21T02:09:16.720063+0000","last_fullsized":"2024-09-21T02:09:16.720063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:38.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722226+0000","last_change":"2024-09-21T02:09:11.558074+0000","last_active":"2024-09-21T02:09:16.722226+0000","last_peered":"2024-09-21T02:09:16.722226+0000","last_clean":"2024-09-21T02:09:16.722226+0000","last_became_active":"2024-09-21T02:09:11.557270+0000","last_became_peered":"2024-09-21T02:09:11.557270+0000","last_unstale":"2024-09-21T02:09:16.722226+0000","last_undegraded":"2024-09-21T02:09:16.722226+0000","last_fullsized":"2024-09-21T02:09:16.722226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:29.728033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590847+0000","last_change":"2024-09-21T02:09:10.540346+0000","last_active":"2024-09-21T02:09:16.590847+0000","last_peered":"2024-09-21T02:09:16.590847+0000","last_clean":"2024-09-21T02:09:16.590847+0000","last_became_active":"2024-09-21T02:09:10.539632+0000","last_became_peered":"2024-09-21T02:09:10.539632+0000","last_unstale":"2024-09-21T02:09:16.590847+0000","last_undegraded":"2024-09-21T02:09:16.590847+0000","last_fullsized":"2024-09-21T02:09:16.590847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722704+0000","last_change":"2024-09-21T02:09:15.636027+0000","last_active":"2024-09-21T02:09:16.722704+0000","last_peered":"2024-09-21T02:09:16.722704+0000","last_clean":"2024-09-21T02:09:16.722704+0000","last_became_active":"2024-09-21T02:09:15.635644+0000","last_became_peered":"2024-09-21T02:09:15.635644+0000","last_unstale":"2024-09-21T02:09:16.722704+0000","last_undegraded":"2024-09-21T02:09:16.722704+0000","last_fullsized":"2024-09-21T02:09:16.722704+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:46.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589275+0000","last_change":"2024-09-21T02:09:11.568395+0000","last_active":"2024-09-21T02:09:16.589275+0000","last_peered":"2024-09-21T02:09:16.589275+0000","last_clean":"2024-09-21T02:09:16.589275+0000","last_became_active":"2024-09-21T02:09:11.567951+0000","last_became_peered":"2024-09-21T02:09:11.567951+0000","last_unstale":"2024-09-21T02:09:16.589275+0000","last_undegraded":"2024-09-21T02:09:16.589275+0000","last_fullsized":"2024-09-21T02:09:16.589275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:38.502107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724904+0000","last_change":"2024-09-21T02:09:10.542137+0000","last_active":"2024-09-21T02:09:16.724904+0000","last_peered":"2024-09-21T02:09:16.724904+0000","last_clean":"2024-09-21T02:09:16.724904+0000","last_became_active":"2024-09-21T02:09:10.541324+0000","last_became_peered":"2024-09-21T02:09:10.541324+0000","last_unstale":"2024-09-21T02:09:16.724904+0000","last_undegraded":"2024-09-21T02:09:16.724904+0000","last_fullsized":"2024-09-21T02:09:16.724904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591215+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:09:16.591215+0000","last_peered":"2024-09-21T02:09:16.591215+0000","last_clean":"2024-09-21T02:09:16.591215+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:09:16.591215+0000","last_undegraded":"2024-09-21T02:09:16.591215+0000","last_fullsized":"2024-09-21T02:09:16.591215+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725602+0000","last_change":"2024-09-21T02:09:15.613320+0000","last_active":"2024-09-21T02:09:16.725602+0000","last_peered":"2024-09-21T02:09:16.725602+0000","last_clean":"2024-09-21T02:09:16.725602+0000","last_became_active":"2024-09-21T02:09:15.612050+0000","last_became_peered":"2024-09-21T02:09:15.612050+0000","last_unstale":"2024-09-21T02:09:16.725602+0000","last_undegraded":"2024-09-21T02:09:16.725602+0000","last_fullsized":"2024-09-21T02:09:16.725602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:54.519499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592039+0000","last_change":"2024-09-21T02:09:11.571228+0000","last_active":"2024-09-21T02:09:16.592039+0000","last_peered":"2024-09-21T02:09:16.592039+0000","last_clean":"2024-09-21T02:09:16.592039+0000","last_became_active":"2024-09-21T02:09:11.570627+0000","last_became_peered":"2024-09-21T02:09:11.570627+0000","last_unstale":"2024-09-21T02:09:16.592039+0000","last_undegraded":"2024-09-21T02:09:16.592039+0000","last_fullsized":"2024-09-21T02:09:16.592039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.513856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721976+0000","last_change":"2024-09-21T02:09:10.530153+0000","last_active":"2024-09-21T02:09:16.721976+0000","last_peered":"2024-09-21T02:09:16.721976+0000","last_clean":"2024-09-21T02:09:16.721976+0000","last_became_active":"2024-09-21T02:09:10.529650+0000","last_became_peered":"2024-09-21T02:09:10.529650+0000","last_unstale":"2024-09-21T02:09:16.721976+0000","last_undegraded":"2024-09-21T02:09:16.721976+0000","last_fullsized":"2024-09-21T02:09:16.721976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724037+0000","last_change":"2024-09-21T02:09:15.609642+0000","last_active":"2024-09-21T02:09:16.724037+0000","last_peered":"2024-09-21T02:09:16.724037+0000","last_clean":"2024-09-21T02:09:16.724037+0000","last_became_active":"2024-09-21T02:09:15.604449+0000","last_became_peered":"2024-09-21T02:09:15.604449+0000","last_unstale":"2024-09-21T02:09:16.724037+0000","last_undegraded":"2024-09-21T02:09:16.724037+0000","last_fullsized":"2024-09-21T02:09:16.724037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:55.894914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724291+0000","last_change":"2024-09-21T02:09:11.554539+0000","last_active":"2024-09-21T02:09:16.724291+0000","last_peered":"2024-09-21T02:09:16.724291+0000","last_clean":"2024-09-21T02:09:16.724291+0000","last_became_active":"2024-09-21T02:09:11.553712+0000","last_became_peered":"2024-09-21T02:09:11.553712+0000","last_unstale":"2024-09-21T02:09:16.724291+0000","last_undegraded":"2024-09-21T02:09:16.724291+0000","last_fullsized":"2024-09-21T02:09:16.724291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.229624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719880+0000","last_change":"2024-09-21T02:09:10.545134+0000","last_active":"2024-09-21T02:09:16.719880+0000","last_peered":"2024-09-21T02:09:16.719880+0000","last_clean":"2024-09-21T02:09:16.719880+0000","last_became_active":"2024-09-21T02:09:10.544792+0000","last_became_peered":"2024-09-21T02:09:10.544792+0000","last_unstale":"2024-09-21T02:09:16.719880+0000","last_undegraded":"2024-09-21T02:09:16.719880+0000","last_fullsized":"2024-09-21T02:09:16.719880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723145+0000","last_change":"2024-09-21T02:09:15.626483+0000","last_active":"2024-09-21T02:09:16.723145+0000","last_peered":"2024-09-21T02:09:16.723145+0000","last_clean":"2024-09-21T02:09:16.723145+0000","last_became_active":"2024-09-21T02:09:15.626132+0000","last_became_peered":"2024-09-21T02:09:15.626132+0000","last_unstale":"2024-09-21T02:09:16.723145+0000","last_undegraded":"2024-09-21T02:09:16.723145+0000","last_fullsized":"2024-09-21T02:09:16.723145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:04.011318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592725+0000","last_change":"2024-09-21T02:09:11.533777+0000","last_active":"2024-09-21T02:09:16.592725+0000","last_peered":"2024-09-21T02:09:16.592725+0000","last_clean":"2024-09-21T02:09:16.592725+0000","last_became_active":"2024-09-21T02:09:11.532928+0000","last_became_peered":"2024-09-21T02:09:11.532928+0000","last_unstale":"2024-09-21T02:09:16.592725+0000","last_undegraded":"2024-09-21T02:09:16.592725+0000","last_fullsized":"2024-09-21T02:09:16.592725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:21.176822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721695+0000","last_change":"2024-09-21T02:09:10.524899+0000","last_active":"2024-09-21T02:09:16.721695+0000","last_peered":"2024-09-21T02:09:16.721695+0000","last_clean":"2024-09-21T02:09:16.721695+0000","last_became_active":"2024-09-21T02:09:10.524447+0000","last_became_peered":"2024-09-21T02:09:10.524447+0000","last_unstale":"2024-09-21T02:09:16.721695+0000","last_undegraded":"2024-09-21T02:09:16.721695+0000","last_fullsized":"2024-09-21T02:09:16.721695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723182+0000","last_change":"2024-09-21T02:09:15.623410+0000","last_active":"2024-09-21T02:09:16.723182+0000","last_peered":"2024-09-21T02:09:16.723182+0000","last_clean":"2024-09-21T02:09:16.723182+0000","last_became_active":"2024-09-21T02:09:15.622194+0000","last_became_peered":"2024-09-21T02:09:15.622194+0000","last_unstale":"2024-09-21T02:09:16.723182+0000","last_undegraded":"2024-09-21T02:09:16.723182+0000","last_fullsized":"2024-09-21T02:09:16.723182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594437+0000","last_change":"2024-09-21T02:09:11.531628+0000","last_active":"2024-09-21T02:09:16.594437+0000","last_peered":"2024-09-21T02:09:16.594437+0000","last_clean":"2024-09-21T02:09:16.594437+0000","last_became_active":"2024-09-21T02:09:11.522487+0000","last_became_peered":"2024-09-21T02:09:11.522487+0000","last_unstale":"2024-09-21T02:09:16.594437+0000","last_undegraded":"2024-09-21T02:09:16.594437+0000","last_fullsized":"2024-09-21T02:09:16.594437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:31.403053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595653+0000","last_change":"2024-09-21T02:09:10.526000+0000","last_active":"2024-09-21T02:09:16.595653+0000","last_peered":"2024-09-21T02:09:16.595653+0000","last_clean":"2024-09-21T02:09:16.595653+0000","last_became_active":"2024-09-21T02:09:10.525267+0000","last_became_peered":"2024-09-21T02:09:10.525267+0000","last_unstale":"2024-09-21T02:09:16.595653+0000","last_undegraded":"2024-09-21T02:09:16.595653+0000","last_fullsized":"2024-09-21T02:09:16.595653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:32.448144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592641+0000","last_change":"2024-09-21T02:09:15.626495+0000","last_active":"2024-09-21T02:09:16.592641+0000","last_peered":"2024-09-21T02:09:16.592641+0000","last_clean":"2024-09-21T02:09:16.592641+0000","last_became_active":"2024-09-21T02:09:15.625151+0000","last_became_peered":"2024-09-21T02:09:15.625151+0000","last_unstale":"2024-09-21T02:09:16.592641+0000","last_undegraded":"2024-09-21T02:09:16.592641+0000","last_fullsized":"2024-09-21T02:09:16.592641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:23.340983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591341+0000","last_change":"2024-09-21T02:09:11.521241+0000","last_active":"2024-09-21T02:09:16.591341+0000","last_peered":"2024-09-21T02:09:16.591341+0000","last_clean":"2024-09-21T02:09:16.591341+0000","last_became_active":"2024-09-21T02:09:11.519883+0000","last_became_peered":"2024-09-21T02:09:11.519883+0000","last_unstale":"2024-09-21T02:09:16.591341+0000","last_undegraded":"2024-09-21T02:09:16.591341+0000","last_fullsized":"2024-09-21T02:09:16.591341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:26.260780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720026+0000","last_change":"2024-09-21T02:09:10.520638+0000","last_active":"2024-09-21T02:09:16.720026+0000","last_peered":"2024-09-21T02:09:16.720026+0000","last_clean":"2024-09-21T02:09:16.720026+0000","last_became_active":"2024-09-21T02:09:10.520131+0000","last_became_peered":"2024-09-21T02:09:10.520131+0000","last_unstale":"2024-09-21T02:09:16.720026+0000","last_undegraded":"2024-09-21T02:09:16.720026+0000","last_fullsized":"2024-09-21T02:09:16.720026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723034+0000","last_change":"2024-09-21T02:09:15.610523+0000","last_active":"2024-09-21T02:09:16.723034+0000","last_peered":"2024-09-21T02:09:16.723034+0000","last_clean":"2024-09-21T02:09:16.723034+0000","last_became_active":"2024-09-21T02:09:15.609441+0000","last_became_peered":"2024-09-21T02:09:15.609441+0000","last_unstale":"2024-09-21T02:09:16.723034+0000","last_undegraded":"2024-09-21T02:09:16.723034+0000","last_fullsized":"2024-09-21T02:09:16.723034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:36.011939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722864+0000","last_change":"2024-09-21T02:09:11.561816+0000","last_active":"2024-09-21T02:09:16.722864+0000","last_peered":"2024-09-21T02:09:16.722864+0000","last_clean":"2024-09-21T02:09:16.722864+0000","last_became_active":"2024-09-21T02:09:11.561231+0000","last_became_peered":"2024-09-21T02:09:11.561231+0000","last_unstale":"2024-09-21T02:09:16.722864+0000","last_undegraded":"2024-09-21T02:09:16.722864+0000","last_fullsized":"2024-09-21T02:09:16.722864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:38.366733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723173+0000","last_change":"2024-09-21T02:09:10.550508+0000","last_active":"2024-09-21T02:09:16.723173+0000","last_peered":"2024-09-21T02:09:16.723173+0000","last_clean":"2024-09-21T02:09:16.723173+0000","last_became_active":"2024-09-21T02:09:10.549913+0000","last_became_peered":"2024-09-21T02:09:10.549913+0000","last_unstale":"2024-09-21T02:09:16.723173+0000","last_undegraded":"2024-09-21T02:09:16.723173+0000","last_fullsized":"2024-09-21T02:09:16.723173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:15.201315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591175+0000","last_change":"2024-09-21T02:09:15.618146+0000","last_active":"2024-09-21T02:09:16.591175+0000","last_peered":"2024-09-21T02:09:16.591175+0000","last_clean":"2024-09-21T02:09:16.591175+0000","last_became_active":"2024-09-21T02:09:15.617195+0000","last_became_peered":"2024-09-21T02:09:15.617195+0000","last_unstale":"2024-09-21T02:09:16.591175+0000","last_undegraded":"2024-09-21T02:09:16.591175+0000","last_fullsized":"2024-09-21T02:09:16.591175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.452372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592327+0000","last_change":"2024-09-21T02:09:11.520992+0000","last_active":"2024-09-21T02:09:16.592327+0000","last_peered":"2024-09-21T02:09:16.592327+0000","last_clean":"2024-09-21T02:09:16.592327+0000","last_became_active":"2024-09-21T02:09:11.519892+0000","last_became_peered":"2024-09-21T02:09:11.519892+0000","last_unstale":"2024-09-21T02:09:16.592327+0000","last_undegraded":"2024-09-21T02:09:16.592327+0000","last_fullsized":"2024-09-21T02:09:16.592327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:34.673193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724869+0000","last_change":"2024-09-21T02:09:10.543820+0000","last_active":"2024-09-21T02:09:16.724869+0000","last_peered":"2024-09-21T02:09:16.724869+0000","last_clean":"2024-09-21T02:09:16.724869+0000","last_became_active":"2024-09-21T02:09:10.542944+0000","last_became_peered":"2024-09-21T02:09:10.542944+0000","last_unstale":"2024-09-21T02:09:16.724869+0000","last_undegraded":"2024-09-21T02:09:16.724869+0000","last_fullsized":"2024-09-21T02:09:16.724869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:59.409264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590058+0000","last_change":"2024-09-21T02:09:15.612341+0000","last_active":"2024-09-21T02:09:16.590058+0000","last_peered":"2024-09-21T02:09:16.590058+0000","last_clean":"2024-09-21T02:09:16.590058+0000","last_became_active":"2024-09-21T02:09:15.611864+0000","last_became_peered":"2024-09-21T02:09:15.611864+0000","last_unstale":"2024-09-21T02:09:16.590058+0000","last_undegraded":"2024-09-21T02:09:16.590058+0000","last_fullsized":"2024-09-21T02:09:16.590058+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:25.484143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723926+0000","last_change":"2024-09-21T02:09:11.557442+0000","last_active":"2024-09-21T02:09:16.723926+0000","last_peered":"2024-09-21T02:09:16.723926+0000","last_clean":"2024-09-21T02:09:16.723926+0000","last_became_active":"2024-09-21T02:09:11.557119+0000","last_became_peered":"2024-09-21T02:09:11.557119+0000","last_unstale":"2024-09-21T02:09:16.723926+0000","last_undegraded":"2024-09-21T02:09:16.723926+0000","last_fullsized":"2024-09-21T02:09:16.723926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723410+0000","last_change":"2024-09-21T02:09:10.530343+0000","last_active":"2024-09-21T02:09:16.723410+0000","last_peered":"2024-09-21T02:09:16.723410+0000","last_clean":"2024-09-21T02:09:16.723410+0000","last_became_active":"2024-09-21T02:09:10.529930+0000","last_became_peered":"2024-09-21T02:09:10.529930+0000","last_unstale":"2024-09-21T02:09:16.723410+0000","last_undegraded":"2024-09-21T02:09:16.723410+0000","last_fullsized":"2024-09-21T02:09:16.723410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725837+0000","last_change":"2024-09-21T02:09:15.635216+0000","last_active":"2024-09-21T02:09:16.725837+0000","last_peered":"2024-09-21T02:09:16.725837+0000","last_clean":"2024-09-21T02:09:16.725837+0000","last_became_active":"2024-09-21T02:09:15.626937+0000","last_became_peered":"2024-09-21T02:09:15.626937+0000","last_unstale":"2024-09-21T02:09:16.725837+0000","last_undegraded":"2024-09-21T02:09:16.725837+0000","last_fullsized":"2024-09-21T02:09:16.725837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591747+0000","last_change":"2024-09-21T02:09:11.538575+0000","last_active":"2024-09-21T02:09:16.591747+0000","last_peered":"2024-09-21T02:09:16.591747+0000","last_clean":"2024-09-21T02:09:16.591747+0000","last_became_active":"2024-09-21T02:09:11.535561+0000","last_became_peered":"2024-09-21T02:09:11.535561+0000","last_unstale":"2024-09-21T02:09:16.591747+0000","last_undegraded":"2024-09-21T02:09:16.591747+0000","last_fullsized":"2024-09-21T02:09:16.591747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:58.627736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:22.534320+0000","last_change":"2024-09-21T02:09:10.546712+0000","last_active":"2024-09-21T02:09:22.534320+0000","last_peered":"2024-09-21T02:09:22.534320+0000","last_clean":"2024-09-21T02:09:22.534320+0000","last_became_active":"2024-09-21T02:09:10.541968+0000","last_became_peered":"2024-09-21T02:09:10.541968+0000","last_unstale":"2024-09-21T02:09:22.534320+0000","last_undegraded":"2024-09-21T02:09:22.534320+0000","last_fullsized":"2024-09-21T02:09:22.534320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720210+0000","last_change":"2024-09-21T02:09:15.616116+0000","last_active":"2024-09-21T02:09:16.720210+0000","last_peered":"2024-09-21T02:09:16.720210+0000","last_clean":"2024-09-21T02:09:16.720210+0000","last_became_active":"2024-09-21T02:09:15.615337+0000","last_became_peered":"2024-09-21T02:09:15.615337+0000","last_unstale":"2024-09-21T02:09:16.720210+0000","last_undegraded":"2024-09-21T02:09:16.720210+0000","last_fullsized":"2024-09-21T02:09:16.720210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:35.472712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594788+0000","last_change":"2024-09-21T02:09:11.520271+0000","last_active":"2024-09-21T02:09:16.594788+0000","last_peered":"2024-09-21T02:09:16.594788+0000","last_clean":"2024-09-21T02:09:16.594788+0000","last_became_active":"2024-09-21T02:09:11.518967+0000","last_became_peered":"2024-09-21T02:09:11.518967+0000","last_unstale":"2024-09-21T02:09:16.594788+0000","last_undegraded":"2024-09-21T02:09:16.594788+0000","last_fullsized":"2024-09-21T02:09:16.594788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719861+0000","last_change":"2024-09-21T02:09:10.536568+0000","last_active":"2024-09-21T02:09:16.719861+0000","last_peered":"2024-09-21T02:09:16.719861+0000","last_clean":"2024-09-21T02:09:16.719861+0000","last_became_active":"2024-09-21T02:09:10.536266+0000","last_became_peered":"2024-09-21T02:09:10.536266+0000","last_unstale":"2024-09-21T02:09:16.719861+0000","last_undegraded":"2024-09-21T02:09:16.719861+0000","last_fullsized":"2024-09-21T02:09:16.719861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595233+0000","last_change":"2024-09-21T02:09:15.637262+0000","last_active":"2024-09-21T02:09:16.595233+0000","last_peered":"2024-09-21T02:09:16.595233+0000","last_clean":"2024-09-21T02:09:16.595233+0000","last_became_active":"2024-09-21T02:09:15.636305+0000","last_became_peered":"2024-09-21T02:09:15.636305+0000","last_unstale":"2024-09-21T02:09:16.595233+0000","last_undegraded":"2024-09-21T02:09:16.595233+0000","last_fullsized":"2024-09-21T02:09:16.595233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:36.447874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723352+0000","last_change":"2024-09-21T02:09:11.562979+0000","last_active":"2024-09-21T02:09:16.723352+0000","last_peered":"2024-09-21T02:09:16.723352+0000","last_clean":"2024-09-21T02:09:16.723352+0000","last_became_active":"2024-09-21T02:09:11.559009+0000","last_became_peered":"2024-09-21T02:09:11.559009+0000","last_unstale":"2024-09-21T02:09:16.723352+0000","last_undegraded":"2024-09-21T02:09:16.723352+0000","last_fullsized":"2024-09-21T02:09:16.723352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:27.296832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725417+0000","last_change":"2024-09-21T02:09:10.546521+0000","last_active":"2024-09-21T02:09:16.725417+0000","last_peered":"2024-09-21T02:09:16.725417+0000","last_clean":"2024-09-21T02:09:16.725417+0000","last_became_active":"2024-09-21T02:09:10.545864+0000","last_became_peered":"2024-09-21T02:09:10.545864+0000","last_unstale":"2024-09-21T02:09:16.725417+0000","last_undegraded":"2024-09-21T02:09:16.725417+0000","last_fullsized":"2024-09-21T02:09:16.725417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589335+0000","last_change":"2024-09-21T02:09:15.616179+0000","last_active":"2024-09-21T02:09:16.589335+0000","last_peered":"2024-09-21T02:09:16.589335+0000","last_clean":"2024-09-21T02:09:16.589335+0000","last_became_active":"2024-09-21T02:09:15.615344+0000","last_became_peered":"2024-09-21T02:09:15.615344+0000","last_unstale":"2024-09-21T02:09:16.589335+0000","last_undegraded":"2024-09-21T02:09:16.589335+0000","last_fullsized":"2024-09-21T02:09:16.589335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.519680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725123+0000","last_change":"2024-09-21T02:09:11.582832+0000","last_active":"2024-09-21T02:09:16.725123+0000","last_peered":"2024-09-21T02:09:16.725123+0000","last_clean":"2024-09-21T02:09:16.725123+0000","last_became_active":"2024-09-21T02:09:11.582121+0000","last_became_peered":"2024-09-21T02:09:11.582121+0000","last_unstale":"2024-09-21T02:09:16.725123+0000","last_undegraded":"2024-09-21T02:09:16.725123+0000","last_fullsized":"2024-09-21T02:09:16.725123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.829896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726463+0000","last_change":"2024-09-21T02:09:10.556671+0000","last_active":"2024-09-21T02:09:16.726463+0000","last_peered":"2024-09-21T02:09:16.726463+0000","last_clean":"2024-09-21T02:09:16.726463+0000","last_became_active":"2024-09-21T02:09:10.555875+0000","last_became_peered":"2024-09-21T02:09:10.555875+0000","last_unstale":"2024-09-21T02:09:16.726463+0000","last_undegraded":"2024-09-21T02:09:16.726463+0000","last_fullsized":"2024-09-21T02:09:16.726463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588965+0000","last_change":"2024-09-21T02:09:15.623197+0000","last_active":"2024-09-21T02:09:16.588965+0000","last_peered":"2024-09-21T02:09:16.588965+0000","last_clean":"2024-09-21T02:09:16.588965+0000","last_became_active":"2024-09-21T02:09:15.622340+0000","last_became_peered":"2024-09-21T02:09:15.622340+0000","last_unstale":"2024-09-21T02:09:16.588965+0000","last_undegraded":"2024-09-21T02:09:16.588965+0000","last_fullsized":"2024-09-21T02:09:16.588965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722347+0000","last_change":"2024-09-21T02:09:15.636227+0000","last_active":"2024-09-21T02:09:16.722347+0000","last_peered":"2024-09-21T02:09:16.722347+0000","last_clean":"2024-09-21T02:09:16.722347+0000","last_became_active":"2024-09-21T02:09:15.635814+0000","last_became_peered":"2024-09-21T02:09:15.635814+0000","last_unstale":"2024-09-21T02:09:16.722347+0000","last_undegraded":"2024-09-21T02:09:16.722347+0000","last_fullsized":"2024-09-21T02:09:16.722347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:37.103933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719830+0000","last_change":"2024-09-21T02:09:11.531143+0000","last_active":"2024-09-21T02:09:16.719830+0000","last_peered":"2024-09-21T02:09:16.719830+0000","last_clean":"2024-09-21T02:09:16.719830+0000","last_became_active":"2024-09-21T02:09:11.530348+0000","last_became_peered":"2024-09-21T02:09:11.530348+0000","last_unstale":"2024-09-21T02:09:16.719830+0000","last_undegraded":"2024-09-21T02:09:16.719830+0000","last_fullsized":"2024-09-21T02:09:16.719830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:21.823287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723931+0000","last_change":"2024-09-21T02:09:10.538525+0000","last_active":"2024-09-21T02:09:16.723931+0000","last_peered":"2024-09-21T02:09:16.723931+0000","last_clean":"2024-09-21T02:09:16.723931+0000","last_became_active":"2024-09-21T02:09:10.537466+0000","last_became_peered":"2024-09-21T02:09:10.537466+0000","last_unstale":"2024-09-21T02:09:16.723931+0000","last_undegraded":"2024-09-21T02:09:16.723931+0000","last_fullsized":"2024-09-21T02:09:16.723931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726815+0000","last_change":"2024-09-21T02:09:15.625976+0000","last_active":"2024-09-21T02:09:16.726815+0000","last_peered":"2024-09-21T02:09:16.726815+0000","last_clean":"2024-09-21T02:09:16.726815+0000","last_became_active":"2024-09-21T02:09:15.624692+0000","last_became_peered":"2024-09-21T02:09:15.624692+0000","last_unstale":"2024-09-21T02:09:16.726815+0000","last_undegraded":"2024-09-21T02:09:16.726815+0000","last_fullsized":"2024-09-21T02:09:16.726815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:18.311891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588890+0000","last_change":"2024-09-21T02:09:11.544871+0000","last_active":"2024-09-21T02:09:16.588890+0000","last_peered":"2024-09-21T02:09:16.588890+0000","last_clean":"2024-09-21T02:09:16.588890+0000","last_became_active":"2024-09-21T02:09:11.543403+0000","last_became_peered":"2024-09-21T02:09:11.543403+0000","last_unstale":"2024-09-21T02:09:16.588890+0000","last_undegraded":"2024-09-21T02:09:16.588890+0000","last_fullsized":"2024-09-21T02:09:16.588890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589183+0000","last_change":"2024-09-21T02:09:10.539555+0000","last_active":"2024-09-21T02:09:16.589183+0000","last_peered":"2024-09-21T02:09:16.589183+0000","last_clean":"2024-09-21T02:09:16.589183+0000","last_became_active":"2024-09-21T02:09:10.539163+0000","last_became_peered":"2024-09-21T02:09:10.539163+0000","last_unstale":"2024-09-21T02:09:16.589183+0000","last_undegraded":"2024-09-21T02:09:16.589183+0000","last_fullsized":"2024-09-21T02:09:16.589183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724509+0000","last_change":"2024-09-21T02:09:10.550879+0000","last_active":"2024-09-21T02:09:16.724509+0000","last_peered":"2024-09-21T02:09:16.724509+0000","last_clean":"2024-09-21T02:09:16.724509+0000","last_became_active":"2024-09-21T02:09:10.550147+0000","last_became_peered":"2024-09-21T02:09:10.550147+0000","last_unstale":"2024-09-21T02:09:16.724509+0000","last_undegraded":"2024-09-21T02:09:16.724509+0000","last_fullsized":"2024-09-21T02:09:16.724509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723322+0000","last_change":"2024-09-21T02:09:11.564520+0000","last_active":"2024-09-21T02:09:16.723322+0000","last_peered":"2024-09-21T02:09:16.723322+0000","last_clean":"2024-09-21T02:09:16.723322+0000","last_became_active":"2024-09-21T02:09:11.563870+0000","last_became_peered":"2024-09-21T02:09:11.563870+0000","last_unstale":"2024-09-21T02:09:16.723322+0000","last_undegraded":"2024-09-21T02:09:16.723322+0000","last_fullsized":"2024-09-21T02:09:16.723322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:20.175775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721646+0000","last_change":"2024-09-21T02:09:15.623267+0000","last_active":"2024-09-21T02:09:16.721646+0000","last_peered":"2024-09-21T02:09:16.721646+0000","last_clean":"2024-09-21T02:09:16.721646+0000","last_became_active":"2024-09-21T02:09:15.622816+0000","last_became_peered":"2024-09-21T02:09:15.622816+0000","last_unstale":"2024-09-21T02:09:16.721646+0000","last_undegraded":"2024-09-21T02:09:16.721646+0000","last_fullsized":"2024-09-21T02:09:16.721646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:11.398708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589624+0000","last_change":"2024-09-21T02:09:10.545988+0000","last_active":"2024-09-21T02:09:16.589624+0000","last_peered":"2024-09-21T02:09:16.589624+0000","last_clean":"2024-09-21T02:09:16.589624+0000","last_became_active":"2024-09-21T02:09:10.545537+0000","last_became_peered":"2024-09-21T02:09:10.545537+0000","last_unstale":"2024-09-21T02:09:16.589624+0000","last_undegraded":"2024-09-21T02:09:16.589624+0000","last_fullsized":"2024-09-21T02:09:16.589624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592924+0000","last_change":"2024-09-21T02:09:11.527921+0000","last_active":"2024-09-21T02:09:16.592924+0000","last_peered":"2024-09-21T02:09:16.592924+0000","last_clean":"2024-09-21T02:09:16.592924+0000","last_became_active":"2024-09-21T02:09:11.527033+0000","last_became_peered":"2024-09-21T02:09:11.527033+0000","last_unstale":"2024-09-21T02:09:16.592924+0000","last_undegraded":"2024-09-21T02:09:16.592924+0000","last_fullsized":"2024-09-21T02:09:16.592924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590282+0000","last_change":"2024-09-21T02:09:15.608837+0000","last_active":"2024-09-21T02:09:16.590282+0000","last_peered":"2024-09-21T02:09:16.590282+0000","last_clean":"2024-09-21T02:09:16.590282+0000","last_became_active":"2024-09-21T02:09:15.607905+0000","last_became_peered":"2024-09-21T02:09:15.607905+0000","last_unstale":"2024-09-21T02:09:16.590282+0000","last_undegraded":"2024-09-21T02:09:16.590282+0000","last_fullsized":"2024-09-21T02:09:16.590282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:15.347636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724603+0000","last_change":"2024-09-21T02:09:11.562756+0000","last_active":"2024-09-21T02:09:16.724603+0000","last_peered":"2024-09-21T02:09:16.724603+0000","last_clean":"2024-09-21T02:09:16.724603+0000","last_became_active":"2024-09-21T02:09:11.560604+0000","last_became_peered":"2024-09-21T02:09:11.560604+0000","last_unstale":"2024-09-21T02:09:16.724603+0000","last_undegraded":"2024-09-21T02:09:16.724603+0000","last_fullsized":"2024-09-21T02:09:16.724603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719949+0000","last_change":"2024-09-21T02:09:11.566770+0000","last_active":"2024-09-21T02:09:16.719949+0000","last_peered":"2024-09-21T02:09:16.719949+0000","last_clean":"2024-09-21T02:09:16.719949+0000","last_became_active":"2024-09-21T02:09:11.566425+0000","last_became_peered":"2024-09-21T02:09:11.566425+0000","last_unstale":"2024-09-21T02:09:16.719949+0000","last_undegraded":"2024-09-21T02:09:16.719949+0000","last_fullsized":"2024-09-21T02:09:16.719949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724847+0000","last_change":"2024-09-21T02:09:11.572089+0000","last_active":"2024-09-21T02:09:16.724847+0000","last_peered":"2024-09-21T02:09:16.724847+0000","last_clean":"2024-09-21T02:09:16.724847+0000","last_became_active":"2024-09-21T02:09:11.566576+0000","last_became_peered":"2024-09-21T02:09:11.566576+0000","last_unstale":"2024-09-21T02:09:16.724847+0000","last_undegraded":"2024-09-21T02:09:16.724847+0000","last_fullsized":"2024-09-21T02:09:16.724847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:42.706829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723234+0000","last_change":"2024-09-21T02:09:11.564192+0000","last_active":"2024-09-21T02:09:16.723234+0000","last_peered":"2024-09-21T02:09:16.723234+0000","last_clean":"2024-09-21T02:09:16.723234+0000","last_became_active":"2024-09-21T02:09:11.559079+0000","last_became_peered":"2024-09-21T02:09:11.559079+0000","last_unstale":"2024-09-21T02:09:16.723234+0000","last_undegraded":"2024-09-21T02:09:16.723234+0000","last_fullsized":"2024-09-21T02:09:16.723234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590380+0000","last_change":"2024-09-21T02:09:11.540588+0000","last_active":"2024-09-21T02:09:16.590380+0000","last_peered":"2024-09-21T02:09:16.590380+0000","last_clean":"2024-09-21T02:09:16.590380+0000","last_became_active":"2024-09-21T02:09:11.539779+0000","last_became_peered":"2024-09-21T02:09:11.539779+0000","last_unstale":"2024-09-21T02:09:16.590380+0000","last_undegraded":"2024-09-21T02:09:16.590380+0000","last_fullsized":"2024-09-21T02:09:16.590380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:24.826345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719412+0000","last_change":"2024-09-21T02:09:11.530898+0000","last_active":"2024-09-21T02:09:16.719412+0000","last_peered":"2024-09-21T02:09:16.719412+0000","last_clean":"2024-09-21T02:09:16.719412+0000","last_became_active":"2024-09-21T02:09:11.530100+0000","last_became_peered":"2024-09-21T02:09:11.530100+0000","last_unstale":"2024-09-21T02:09:16.719412+0000","last_undegraded":"2024-09-21T02:09:16.719412+0000","last_fullsized":"2024-09-21T02:09:16.719412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726838+0000","last_change":"2024-09-21T02:09:11.549198+0000","last_active":"2024-09-21T02:09:16.726838+0000","last_peered":"2024-09-21T02:09:16.726838+0000","last_clean":"2024-09-21T02:09:16.726838+0000","last_became_active":"2024-09-21T02:09:11.548421+0000","last_became_peered":"2024-09-21T02:09:11.548421+0000","last_unstale":"2024-09-21T02:09:16.726838+0000","last_undegraded":"2024-09-21T02:09:16.726838+0000","last_fullsized":"2024-09-21T02:09:16.726838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:22.505823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590900+0000","last_change":"2024-09-21T02:09:11.552426+0000","last_active":"2024-09-21T02:09:16.590900+0000","last_peered":"2024-09-21T02:09:16.590900+0000","last_clean":"2024-09-21T02:09:16.590900+0000","last_became_active":"2024-09-21T02:09:11.551844+0000","last_became_peered":"2024-09-21T02:09:11.551844+0000","last_unstale":"2024-09-21T02:09:16.590900+0000","last_undegraded":"2024-09-21T02:09:16.590900+0000","last_fullsized":"2024-09-21T02:09:16.590900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723635+0000","last_change":"2024-09-21T02:09:11.566549+0000","last_active":"2024-09-21T02:09:16.723635+0000","last_peered":"2024-09-21T02:09:16.723635+0000","last_clean":"2024-09-21T02:09:16.723635+0000","last_became_active":"2024-09-21T02:09:11.565905+0000","last_became_peered":"2024-09-21T02:09:11.565905+0000","last_unstale":"2024-09-21T02:09:16.723635+0000","last_undegraded":"2024-09-21T02:09:16.723635+0000","last_fullsized":"2024-09-21T02:09:16.723635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.460538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589185+0000","last_change":"2024-09-21T02:09:11.552230+0000","last_active":"2024-09-21T02:09:16.589185+0000","last_peered":"2024-09-21T02:09:16.589185+0000","last_clean":"2024-09-21T02:09:16.589185+0000","last_became_active":"2024-09-21T02:09:11.551873+0000","last_became_peered":"2024-09-21T02:09:11.551873+0000","last_unstale":"2024-09-21T02:09:16.589185+0000","last_undegraded":"2024-09-21T02:09:16.589185+0000","last_fullsized":"2024-09-21T02:09:16.589185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724470+0000","last_change":"2024-09-21T02:09:11.581809+0000","last_active":"2024-09-21T02:09:16.724470+0000","last_peered":"2024-09-21T02:09:16.724470+0000","last_clean":"2024-09-21T02:09:16.724470+0000","last_became_active":"2024-09-21T02:09:11.581010+0000","last_became_peered":"2024-09-21T02:09:11.581010+0000","last_unstale":"2024-09-21T02:09:16.724470+0000","last_undegraded":"2024-09-21T02:09:16.724470+0000","last_fullsized":"2024-09-21T02:09:16.724470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:19.271324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722807+0000","last_change":"2024-09-21T02:09:11.558586+0000","last_active":"2024-09-21T02:09:16.722807+0000","last_peered":"2024-09-21T02:09:16.722807+0000","last_clean":"2024-09-21T02:09:16.722807+0000","last_became_active":"2024-09-21T02:09:11.558126+0000","last_became_peered":"2024-09-21T02:09:11.558126+0000","last_unstale":"2024-09-21T02:09:16.722807+0000","last_undegraded":"2024-09-21T02:09:16.722807+0000","last_fullsized":"2024-09-21T02:09:16.722807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:10.080231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725755+0000","last_change":"2024-09-21T02:09:11.561183+0000","last_active":"2024-09-21T02:09:16.725755+0000","last_peered":"2024-09-21T02:09:16.725755+0000","last_clean":"2024-09-21T02:09:16.725755+0000","last_became_active":"2024-09-21T02:09:11.560260+0000","last_became_peered":"2024-09-21T02:09:11.560260+0000","last_unstale":"2024-09-21T02:09:16.725755+0000","last_undegraded":"2024-09-21T02:09:16.725755+0000","last_fullsized":"2024-09-21T02:09:16.725755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593415+0000","last_change":"2024-09-21T02:09:11.540272+0000","last_active":"2024-09-21T02:09:16.593415+0000","last_peered":"2024-09-21T02:09:16.593415+0000","last_clean":"2024-09-21T02:09:16.593415+0000","last_became_active":"2024-09-21T02:09:11.539409+0000","last_became_peered":"2024-09-21T02:09:11.539409+0000","last_unstale":"2024-09-21T02:09:16.593415+0000","last_undegraded":"2024-09-21T02:09:16.593415+0000","last_fullsized":"2024-09-21T02:09:16.593415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:28.820372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721399+0000","last_change":"2024-09-21T02:09:11.544995+0000","last_active":"2024-09-21T02:09:16.721399+0000","last_peered":"2024-09-21T02:09:16.721399+0000","last_clean":"2024-09-21T02:09:16.721399+0000","last_became_active":"2024-09-21T02:09:11.544306+0000","last_became_peered":"2024-09-21T02:09:11.544306+0000","last_unstale":"2024-09-21T02:09:16.721399+0000","last_undegraded":"2024-09-21T02:09:16.721399+0000","last_fullsized":"2024-09-21T02:09:16.721399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725507+0000","last_change":"2024-09-21T02:09:11.582210+0000","last_active":"2024-09-21T02:09:16.725507+0000","last_peered":"2024-09-21T02:09:16.725507+0000","last_clean":"2024-09-21T02:09:16.725507+0000","last_became_active":"2024-09-21T02:09:11.581734+0000","last_became_peered":"2024-09-21T02:09:11.581734+0000","last_unstale":"2024-09-21T02:09:16.725507+0000","last_undegraded":"2024-09-21T02:09:16.725507+0000","last_fullsized":"2024-09-21T02:09:16.725507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:01.926437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723576+0000","last_change":"2024-09-21T02:09:11.574300+0000","last_active":"2024-09-21T02:09:16.723576+0000","last_peered":"2024-09-21T02:09:16.723576+0000","last_clean":"2024-09-21T02:09:16.723576+0000","last_became_active":"2024-09-21T02:09:11.573443+0000","last_became_peered":"2024-09-21T02:09:11.573443+0000","last_unstale":"2024-09-21T02:09:16.723576+0000","last_undegraded":"2024-09-21T02:09:16.723576+0000","last_fullsized":"2024-09-21T02:09:16.723576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720120+0000","last_change":"2024-09-21T02:09:11.518238+0000","last_active":"2024-09-21T02:09:16.720120+0000","last_peered":"2024-09-21T02:09:16.720120+0000","last_clean":"2024-09-21T02:09:16.720120+0000","last_became_active":"2024-09-21T02:09:11.517379+0000","last_became_peered":"2024-09-21T02:09:11.517379+0000","last_unstale":"2024-09-21T02:09:16.720120+0000","last_undegraded":"2024-09-21T02:09:16.720120+0000","last_fullsized":"2024-09-21T02:09:16.720120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723824+0000","last_change":"2024-09-21T02:09:11.536912+0000","last_active":"2024-09-21T02:09:16.723824+0000","last_peered":"2024-09-21T02:09:16.723824+0000","last_clean":"2024-09-21T02:09:16.723824+0000","last_became_active":"2024-09-21T02:09:11.535607+0000","last_became_peered":"2024-09-21T02:09:11.535607+0000","last_unstale":"2024-09-21T02:09:16.723824+0000","last_undegraded":"2024-09-21T02:09:16.723824+0000","last_fullsized":"2024-09-21T02:09:16.723824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589439+0000","last_change":"2024-09-21T02:09:11.546912+0000","last_active":"2024-09-21T02:09:16.589439+0000","last_peered":"2024-09-21T02:09:16.589439+0000","last_clean":"2024-09-21T02:09:16.589439+0000","last_became_active":"2024-09-21T02:09:11.543438+0000","last_became_peered":"2024-09-21T02:09:11.543438+0000","last_unstale":"2024-09-21T02:09:16.589439+0000","last_undegraded":"2024-09-21T02:09:16.589439+0000","last_fullsized":"2024-09-21T02:09:16.589439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595165+0000","last_change":"2024-09-21T02:09:11.547068+0000","last_active":"2024-09-21T02:09:16.595165+0000","last_peered":"2024-09-21T02:09:16.595165+0000","last_clean":"2024-09-21T02:09:16.595165+0000","last_became_active":"2024-09-21T02:09:11.545661+0000","last_became_peered":"2024-09-21T02:09:11.545661+0000","last_unstale":"2024-09-21T02:09:16.595165+0000","last_undegraded":"2024-09-21T02:09:16.595165+0000","last_fullsized":"2024-09-21T02:09:16.595165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:00.421056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725978+0000","last_change":"2024-09-21T02:09:11.571193+0000","last_active":"2024-09-21T02:09:16.725978+0000","last_peered":"2024-09-21T02:09:16.725978+0000","last_clean":"2024-09-21T02:09:16.725978+0000","last_became_active":"2024-09-21T02:09:11.563913+0000","last_became_peered":"2024-09-21T02:09:11.563913+0000","last_unstale":"2024-09-21T02:09:16.725978+0000","last_undegraded":"2024-09-21T02:09:16.725978+0000","last_fullsized":"2024-09-21T02:09:16.725978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726661+0000","last_change":"2024-09-21T02:09:11.564018+0000","last_active":"2024-09-21T02:09:16.726661+0000","last_peered":"2024-09-21T02:09:16.726661+0000","last_clean":"2024-09-21T02:09:16.726661+0000","last_became_active":"2024-09-21T02:09:11.563374+0000","last_became_peered":"2024-09-21T02:09:11.563374+0000","last_unstale":"2024-09-21T02:09:16.726661+0000","last_undegraded":"2024-09-21T02:09:16.726661+0000","last_fullsized":"2024-09-21T02:09:16.726661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592542+0000","last_change":"2024-09-21T02:09:11.571561+0000","last_active":"2024-09-21T02:09:16.592542+0000","last_peered":"2024-09-21T02:09:16.592542+0000","last_clean":"2024-09-21T02:09:16.592542+0000","last_became_active":"2024-09-21T02:09:11.570942+0000","last_became_peered":"2024-09-21T02:09:11.570942+0000","last_unstale":"2024-09-21T02:09:16.592542+0000","last_undegraded":"2024-09-21T02:09:16.592542+0000","last_fullsized":"2024-09-21T02:09:16.592542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:07.001076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588463+0000","last_change":"2024-09-21T02:09:11.568051+0000","last_active":"2024-09-21T02:09:16.588463+0000","last_peered":"2024-09-21T02:09:16.588463+0000","last_clean":"2024-09-21T02:09:16.588463+0000","last_became_active":"2024-09-21T02:09:11.567478+0000","last_became_peered":"2024-09-21T02:09:11.567478+0000","last_unstale":"2024-09-21T02:09:16.588463+0000","last_undegraded":"2024-09-21T02:09:16.588463+0000","last_fullsized":"2024-09-21T02:09:16.588463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591178+0000","last_change":"2024-09-21T02:09:11.536755+0000","last_active":"2024-09-21T02:09:16.591178+0000","last_peered":"2024-09-21T02:09:16.591178+0000","last_clean":"2024-09-21T02:09:16.591178+0000","last_became_active":"2024-09-21T02:09:11.535871+0000","last_became_peered":"2024-09-21T02:09:11.535871+0000","last_unstale":"2024-09-21T02:09:16.591178+0000","last_undegraded":"2024-09-21T02:09:16.591178+0000","last_fullsized":"2024-09-21T02:09:16.591178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:00.938047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724464+0000","last_change":"2024-09-21T02:09:11.572153+0000","last_active":"2024-09-21T02:09:16.724464+0000","last_peered":"2024-09-21T02:09:16.724464+0000","last_clean":"2024-09-21T02:09:16.724464+0000","last_became_active":"2024-09-21T02:09:11.567470+0000","last_became_peered":"2024-09-21T02:09:11.567470+0000","last_unstale":"2024-09-21T02:09:16.724464+0000","last_undegraded":"2024-09-21T02:09:16.724464+0000","last_fullsized":"2024-09-21T02:09:16.724464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:24.775708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591826+0000","last_change":"2024-09-21T02:09:11.532323+0000","last_active":"2024-09-21T02:09:16.591826+0000","last_peered":"2024-09-21T02:09:16.591826+0000","last_clean":"2024-09-21T02:09:16.591826+0000","last_became_active":"2024-09-21T02:09:11.531480+0000","last_became_peered":"2024-09-21T02:09:11.531480+0000","last_unstale":"2024-09-21T02:09:16.591826+0000","last_undegraded":"2024-09-21T02:09:16.591826+0000","last_fullsized":"2024-09-21T02:09:16.591826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:40.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723696+0000","last_change":"2024-09-21T02:09:11.575204+0000","last_active":"2024-09-21T02:09:16.723696+0000","last_peered":"2024-09-21T02:09:16.723696+0000","last_clean":"2024-09-21T02:09:16.723696+0000","last_became_active":"2024-09-21T02:09:11.567379+0000","last_became_peered":"2024-09-21T02:09:11.567379+0000","last_unstale":"2024-09-21T02:09:16.723696+0000","last_undegraded":"2024-09-21T02:09:16.723696+0000","last_fullsized":"2024-09-21T02:09:16.723696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:48.813808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722559+0000","last_change":"2024-09-21T02:09:11.540599+0000","last_active":"2024-09-21T02:09:16.722559+0000","last_peered":"2024-09-21T02:09:16.722559+0000","last_clean":"2024-09-21T02:09:16.722559+0000","last_became_active":"2024-09-21T02:09:11.539829+0000","last_became_peered":"2024-09-21T02:09:11.539829+0000","last_unstale":"2024-09-21T02:09:16.722559+0000","last_undegraded":"2024-09-21T02:09:16.722559+0000","last_fullsized":"2024-09-21T02:09:16.722559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:54.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719555+0000","last_change":"2024-09-21T02:09:11.534649+0000","last_active":"2024-09-21T02:09:16.719555+0000","last_peered":"2024-09-21T02:09:16.719555+0000","last_clean":"2024-09-21T02:09:16.719555+0000","last_became_active":"2024-09-21T02:09:11.533392+0000","last_became_peered":"2024-09-21T02:09:11.533392+0000","last_unstale":"2024-09-21T02:09:16.719555+0000","last_undegraded":"2024-09-21T02:09:16.719555+0000","last_fullsized":"2024-09-21T02:09:16.719555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592021+0000","last_change":"2024-09-21T02:09:11.546630+0000","last_active":"2024-09-21T02:09:16.592021+0000","last_peered":"2024-09-21T02:09:16.592021+0000","last_clean":"2024-09-21T02:09:16.592021+0000","last_became_active":"2024-09-21T02:09:11.545498+0000","last_became_peered":"2024-09-21T02:09:11.545498+0000","last_unstale":"2024-09-21T02:09:16.592021+0000","last_undegraded":"2024-09-21T02:09:16.592021+0000","last_fullsized":"2024-09-21T02:09:16.592021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:25.840553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":699032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":108172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":108082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":282624,"data_stored":108242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":699088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:09:33.741 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.912+0000 7f7692c8e640 1 -- 172.21.15.96:0/27918748 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c071210 msgr2=0x7f768c071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.912+0000 7f7692c8e640 1 --2- 172.21.15.96:0/27918748 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c071210 0x7f768c071620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7680009840 tx=0x7f768001c760 comp rx=0 tx=0).stop 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 -- 172.21.15.96:0/27918748 shutdown_connections 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 --2- 172.21.15.96:0/27918748 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c071210 0x7f768c071620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 --2- 172.21.15.96:0/27918748 >> v2:172.21.15.122:3300/0 conn(0x7f768c145140 0x7f768c145550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 -- 172.21.15.96:0/27918748 >> 172.21.15.96:0/27918748 conn(0x7f768c06fdb0 msgr2=0x7f768c0701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 -- 172.21.15.96:0/27918748 shutdown_connections 2024-09-21T02:09:33.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 -- 172.21.15.96:0/27918748 wait complete. 2024-09-21T02:09:33.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 Processor -- start 2024-09-21T02:09:33.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 -- start start 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f768c071210 0x7f768c13c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.916+0000 7f7692c8e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c145140 0x7f768c155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 0x7f768c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f768c169ca0 con 0x7f768c15ff70 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f768c169b20 con 0x7f768c071210 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f768c169e20 con 0x7f768c145140 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 0x7f768c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 0x7f768c156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59710/0 (socket says 172.21.15.96:59710) 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 -- 172.21.15.96:0/1669398047 learned_addr learned my addr 172.21.15.96:0/1669398047 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:33.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7690a2a640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f768c071210 0x7f768c13c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f768bfff640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c145140 0x7f768c155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 -- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c145140 msgr2=0x7f768c155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c145140 0x7f768c155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 -- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f768c071210 msgr2=0x7f768c13c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f768c071210 0x7f768c13c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 -- 172.21.15.96:0/1669398047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c15bf10 con 0x7f768c15ff70 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f769122b640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 0x7f768c156090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c3c0 tx=0x7f767c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c011790 con 0x7f768c15ff70 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f767c017070 con 0x7f768c15ff70 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c1536a0 con 0x7f768c15ff70 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c011c60 con 0x7f768c15ff70 2024-09-21T02:09:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c153c60 con 0x7f768c15ff70 2024-09-21T02:09:33.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c15c180 con 0x7f768c15ff70 2024-09-21T02:09:33.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f767c01c460 con 0x7f768c15ff70 2024-09-21T02:09:33.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f76897fa640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7678073230 0x7f7678075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:33.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.920+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f767c094cc0 con 0x7f768c15ff70 2024-09-21T02:09:33.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.924+0000 7f7690a2a640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7678073230 0x7f7678075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:33.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.924+0000 7f7690a2a640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7678073230 0x7f7678075740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7684009eb0 tx=0x7f7684009500 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:33.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:33.924+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f768c071b60 con 0x7f768c15ff70 2024-09-21T02:09:34.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.084+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f768c13ce90 con 0x7f7678073230 2024-09-21T02:09:34.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.100+0000 7f76897fa640 1 -- 172.21.15.96:0/1669398047 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f768c13ce90 con 0x7f7678073230 2024-09-21T02:09:34.107 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:34.110 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7678073230 msgr2=0x7f7678075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7678073230 0x7f7678075740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7684009eb0 tx=0x7f7684009500 comp rx=0 tx=0).stop 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 msgr2=0x7f768c156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 0x7f768c156090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c3c0 tx=0x7f767c00c7c0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 shutdown_connections 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7678073230 0x7f7678075740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f768c15ff70 0x7f768c156090 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f768c145140 0x7f768c155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 --2- 172.21.15.96:0/1669398047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f768c071210 0x7f768c13c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 >> 172.21.15.96:0/1669398047 conn(0x7f768c06fdb0 msgr2=0x7f768c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 shutdown_connections 2024-09-21T02:09:34.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.104+0000 7f7692c8e640 1 -- 172.21.15.96:0/1669398047 wait complete. 2024-09-21T02:09:34.129 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T02:09:32.505056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223768,"kb_used_data":3368,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750952,"statfs":{"total":773094113280,"available":772864974848,"internally_reserved":0,"allocated":3448832,"data_stored":2054672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005923"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723570+0000","last_change":"2024-09-21T02:09:10.527965+0000","last_active":"2024-09-21T02:09:16.723570+0000","last_peered":"2024-09-21T02:09:16.723570+0000","last_clean":"2024-09-21T02:09:16.723570+0000","last_became_active":"2024-09-21T02:09:10.527508+0000","last_became_peered":"2024-09-21T02:09:10.527508+0000","last_unstale":"2024-09-21T02:09:16.723570+0000","last_undegraded":"2024-09-21T02:09:16.723570+0000","last_fullsized":"2024-09-21T02:09:16.723570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:35.986615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590616+0000","last_change":"2024-09-21T02:09:11.546557+0000","last_active":"2024-09-21T02:09:16.590616+0000","last_peered":"2024-09-21T02:09:16.590616+0000","last_clean":"2024-09-21T02:09:16.590616+0000","last_became_active":"2024-09-21T02:09:11.546158+0000","last_became_peered":"2024-09-21T02:09:11.546158+0000","last_unstale":"2024-09-21T02:09:16.590616+0000","last_undegraded":"2024-09-21T02:09:16.590616+0000","last_fullsized":"2024-09-21T02:09:16.590616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588876+0000","last_change":"2024-09-21T02:09:15.612941+0000","last_active":"2024-09-21T02:09:16.588876+0000","last_peered":"2024-09-21T02:09:16.588876+0000","last_clean":"2024-09-21T02:09:16.588876+0000","last_became_active":"2024-09-21T02:09:15.612114+0000","last_became_peered":"2024-09-21T02:09:15.612114+0000","last_unstale":"2024-09-21T02:09:16.588876+0000","last_undegraded":"2024-09-21T02:09:16.588876+0000","last_fullsized":"2024-09-21T02:09:16.588876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590737+0000","last_change":"2024-09-21T02:09:10.523298+0000","last_active":"2024-09-21T02:09:16.590737+0000","last_peered":"2024-09-21T02:09:16.590737+0000","last_clean":"2024-09-21T02:09:16.590737+0000","last_became_active":"2024-09-21T02:09:10.521768+0000","last_became_peered":"2024-09-21T02:09:10.521768+0000","last_unstale":"2024-09-21T02:09:16.590737+0000","last_undegraded":"2024-09-21T02:09:16.590737+0000","last_fullsized":"2024-09-21T02:09:16.590737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723214+0000","last_change":"2024-09-21T02:09:11.573485+0000","last_active":"2024-09-21T02:09:16.723214+0000","last_peered":"2024-09-21T02:09:16.723214+0000","last_clean":"2024-09-21T02:09:16.723214+0000","last_became_active":"2024-09-21T02:09:11.572854+0000","last_became_peered":"2024-09-21T02:09:11.572854+0000","last_unstale":"2024-09-21T02:09:16.723214+0000","last_undegraded":"2024-09-21T02:09:16.723214+0000","last_fullsized":"2024-09-21T02:09:16.723214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591993+0000","last_change":"2024-09-21T02:09:15.629971+0000","last_active":"2024-09-21T02:09:16.591993+0000","last_peered":"2024-09-21T02:09:16.591993+0000","last_clean":"2024-09-21T02:09:16.591993+0000","last_became_active":"2024-09-21T02:09:15.628916+0000","last_became_peered":"2024-09-21T02:09:15.628916+0000","last_unstale":"2024-09-21T02:09:16.591993+0000","last_undegraded":"2024-09-21T02:09:16.591993+0000","last_fullsized":"2024-09-21T02:09:16.591993+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:16.541985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592632+0000","last_change":"2024-09-21T02:09:10.549868+0000","last_active":"2024-09-21T02:09:16.592632+0000","last_peered":"2024-09-21T02:09:16.592632+0000","last_clean":"2024-09-21T02:09:16.592632+0000","last_became_active":"2024-09-21T02:09:10.549154+0000","last_became_peered":"2024-09-21T02:09:10.549154+0000","last_unstale":"2024-09-21T02:09:16.592632+0000","last_undegraded":"2024-09-21T02:09:16.592632+0000","last_fullsized":"2024-09-21T02:09:16.592632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:42.470809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723996+0000","last_change":"2024-09-21T02:09:11.556559+0000","last_active":"2024-09-21T02:09:16.723996+0000","last_peered":"2024-09-21T02:09:16.723996+0000","last_clean":"2024-09-21T02:09:16.723996+0000","last_became_active":"2024-09-21T02:09:11.555720+0000","last_became_peered":"2024-09-21T02:09:11.555720+0000","last_unstale":"2024-09-21T02:09:16.723996+0000","last_undegraded":"2024-09-21T02:09:16.723996+0000","last_fullsized":"2024-09-21T02:09:16.723996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:43.740023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596031+0000","last_change":"2024-09-21T02:09:15.626432+0000","last_active":"2024-09-21T02:09:16.596031+0000","last_peered":"2024-09-21T02:09:16.596031+0000","last_clean":"2024-09-21T02:09:16.596031+0000","last_became_active":"2024-09-21T02:09:15.620746+0000","last_became_peered":"2024-09-21T02:09:15.620746+0000","last_unstale":"2024-09-21T02:09:16.596031+0000","last_undegraded":"2024-09-21T02:09:16.596031+0000","last_fullsized":"2024-09-21T02:09:16.596031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:46.776489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591909+0000","last_change":"2024-09-21T02:09:10.552140+0000","last_active":"2024-09-21T02:09:16.591909+0000","last_peered":"2024-09-21T02:09:16.591909+0000","last_clean":"2024-09-21T02:09:16.591909+0000","last_became_active":"2024-09-21T02:09:10.551607+0000","last_became_peered":"2024-09-21T02:09:10.551607+0000","last_unstale":"2024-09-21T02:09:16.591909+0000","last_undegraded":"2024-09-21T02:09:16.591909+0000","last_fullsized":"2024-09-21T02:09:16.591909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724056+0000","last_change":"2024-09-21T02:09:11.557870+0000","last_active":"2024-09-21T02:09:16.724056+0000","last_peered":"2024-09-21T02:09:16.724056+0000","last_clean":"2024-09-21T02:09:16.724056+0000","last_became_active":"2024-09-21T02:09:11.556258+0000","last_became_peered":"2024-09-21T02:09:11.556258+0000","last_unstale":"2024-09-21T02:09:16.724056+0000","last_undegraded":"2024-09-21T02:09:16.724056+0000","last_fullsized":"2024-09-21T02:09:16.724056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592325+0000","last_change":"2024-09-21T02:09:15.639499+0000","last_active":"2024-09-21T02:09:16.592325+0000","last_peered":"2024-09-21T02:09:16.592325+0000","last_clean":"2024-09-21T02:09:16.592325+0000","last_became_active":"2024-09-21T02:09:15.638499+0000","last_became_peered":"2024-09-21T02:09:15.638499+0000","last_unstale":"2024-09-21T02:09:16.592325+0000","last_undegraded":"2024-09-21T02:09:16.592325+0000","last_fullsized":"2024-09-21T02:09:16.592325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:22.032011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722630+0000","last_change":"2024-09-21T02:09:11.561488+0000","last_active":"2024-09-21T02:09:16.722630+0000","last_peered":"2024-09-21T02:09:16.722630+0000","last_clean":"2024-09-21T02:09:16.722630+0000","last_became_active":"2024-09-21T02:09:11.560683+0000","last_became_peered":"2024-09-21T02:09:11.560683+0000","last_unstale":"2024-09-21T02:09:16.722630+0000","last_undegraded":"2024-09-21T02:09:16.722630+0000","last_fullsized":"2024-09-21T02:09:16.722630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592616+0000","last_change":"2024-09-21T02:09:10.552547+0000","last_active":"2024-09-21T02:09:16.592616+0000","last_peered":"2024-09-21T02:09:16.592616+0000","last_clean":"2024-09-21T02:09:16.592616+0000","last_became_active":"2024-09-21T02:09:10.552056+0000","last_became_peered":"2024-09-21T02:09:10.552056+0000","last_unstale":"2024-09-21T02:09:16.592616+0000","last_undegraded":"2024-09-21T02:09:16.592616+0000","last_fullsized":"2024-09-21T02:09:16.592616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594025+0000","last_change":"2024-09-21T02:09:15.611360+0000","last_active":"2024-09-21T02:09:16.594025+0000","last_peered":"2024-09-21T02:09:16.594025+0000","last_clean":"2024-09-21T02:09:16.594025+0000","last_became_active":"2024-09-21T02:09:15.609991+0000","last_became_peered":"2024-09-21T02:09:15.609991+0000","last_unstale":"2024-09-21T02:09:16.594025+0000","last_undegraded":"2024-09-21T02:09:16.594025+0000","last_fullsized":"2024-09-21T02:09:16.594025+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:24.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589470+0000","last_change":"2024-09-21T02:09:11.545847+0000","last_active":"2024-09-21T02:09:16.589470+0000","last_peered":"2024-09-21T02:09:16.589470+0000","last_clean":"2024-09-21T02:09:16.589470+0000","last_became_active":"2024-09-21T02:09:11.543606+0000","last_became_peered":"2024-09-21T02:09:11.543606+0000","last_unstale":"2024-09-21T02:09:16.589470+0000","last_undegraded":"2024-09-21T02:09:16.589470+0000","last_fullsized":"2024-09-21T02:09:16.589470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720543+0000","last_change":"2024-09-21T02:09:10.524519+0000","last_active":"2024-09-21T02:09:16.720543+0000","last_peered":"2024-09-21T02:09:16.720543+0000","last_clean":"2024-09-21T02:09:16.720543+0000","last_became_active":"2024-09-21T02:09:10.524074+0000","last_became_peered":"2024-09-21T02:09:10.524074+0000","last_unstale":"2024-09-21T02:09:16.720543+0000","last_undegraded":"2024-09-21T02:09:16.720543+0000","last_fullsized":"2024-09-21T02:09:16.720543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594281+0000","last_change":"2024-09-21T02:09:15.618974+0000","last_active":"2024-09-21T02:09:16.594281+0000","last_peered":"2024-09-21T02:09:16.594281+0000","last_clean":"2024-09-21T02:09:16.594281+0000","last_became_active":"2024-09-21T02:09:15.617774+0000","last_became_peered":"2024-09-21T02:09:15.617774+0000","last_unstale":"2024-09-21T02:09:16.594281+0000","last_undegraded":"2024-09-21T02:09:16.594281+0000","last_fullsized":"2024-09-21T02:09:16.594281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:47.809682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589087+0000","last_change":"2024-09-21T02:09:11.538383+0000","last_active":"2024-09-21T02:09:16.589087+0000","last_peered":"2024-09-21T02:09:16.589087+0000","last_clean":"2024-09-21T02:09:16.589087+0000","last_became_active":"2024-09-21T02:09:11.537569+0000","last_became_peered":"2024-09-21T02:09:11.537569+0000","last_unstale":"2024-09-21T02:09:16.589087+0000","last_undegraded":"2024-09-21T02:09:16.589087+0000","last_fullsized":"2024-09-21T02:09:16.589087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720139+0000","last_change":"2024-09-21T02:09:10.537296+0000","last_active":"2024-09-21T02:09:16.720139+0000","last_peered":"2024-09-21T02:09:16.720139+0000","last_clean":"2024-09-21T02:09:16.720139+0000","last_became_active":"2024-09-21T02:09:10.536913+0000","last_became_peered":"2024-09-21T02:09:10.536913+0000","last_unstale":"2024-09-21T02:09:16.720139+0000","last_undegraded":"2024-09-21T02:09:16.720139+0000","last_fullsized":"2024-09-21T02:09:16.720139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720276+0000","last_change":"2024-09-21T02:09:15.606253+0000","last_active":"2024-09-21T02:09:16.720276+0000","last_peered":"2024-09-21T02:09:16.720276+0000","last_clean":"2024-09-21T02:09:16.720276+0000","last_became_active":"2024-09-21T02:09:15.605847+0000","last_became_peered":"2024-09-21T02:09:15.605847+0000","last_unstale":"2024-09-21T02:09:16.720276+0000","last_undegraded":"2024-09-21T02:09:16.720276+0000","last_fullsized":"2024-09-21T02:09:16.720276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594287+0000","last_change":"2024-09-21T02:09:11.541952+0000","last_active":"2024-09-21T02:09:16.594287+0000","last_peered":"2024-09-21T02:09:16.594287+0000","last_clean":"2024-09-21T02:09:16.594287+0000","last_became_active":"2024-09-21T02:09:11.540560+0000","last_became_peered":"2024-09-21T02:09:11.540560+0000","last_unstale":"2024-09-21T02:09:16.594287+0000","last_undegraded":"2024-09-21T02:09:16.594287+0000","last_fullsized":"2024-09-21T02:09:16.594287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:25.306228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591648+0000","last_change":"2024-09-21T02:09:10.544405+0000","last_active":"2024-09-21T02:09:16.591648+0000","last_peered":"2024-09-21T02:09:16.591648+0000","last_clean":"2024-09-21T02:09:16.591648+0000","last_became_active":"2024-09-21T02:09:10.540771+0000","last_became_peered":"2024-09-21T02:09:10.540771+0000","last_unstale":"2024-09-21T02:09:16.591648+0000","last_undegraded":"2024-09-21T02:09:16.591648+0000","last_fullsized":"2024-09-21T02:09:16.591648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593729+0000","last_change":"2024-09-21T02:09:15.626770+0000","last_active":"2024-09-21T02:09:16.593729+0000","last_peered":"2024-09-21T02:09:16.593729+0000","last_clean":"2024-09-21T02:09:16.593729+0000","last_became_active":"2024-09-21T02:09:15.623532+0000","last_became_peered":"2024-09-21T02:09:15.623532+0000","last_unstale":"2024-09-21T02:09:16.593729+0000","last_undegraded":"2024-09-21T02:09:16.593729+0000","last_fullsized":"2024-09-21T02:09:16.593729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:52.883467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722937+0000","last_change":"2024-09-21T02:09:11.539618+0000","last_active":"2024-09-21T02:09:16.722937+0000","last_peered":"2024-09-21T02:09:16.722937+0000","last_clean":"2024-09-21T02:09:16.722937+0000","last_became_active":"2024-09-21T02:09:11.538801+0000","last_became_peered":"2024-09-21T02:09:11.538801+0000","last_unstale":"2024-09-21T02:09:16.722937+0000","last_undegraded":"2024-09-21T02:09:16.722937+0000","last_fullsized":"2024-09-21T02:09:16.722937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726772+0000","last_change":"2024-09-21T02:09:10.545874+0000","last_active":"2024-09-21T02:09:16.726772+0000","last_peered":"2024-09-21T02:09:16.726772+0000","last_clean":"2024-09-21T02:09:16.726772+0000","last_became_active":"2024-09-21T02:09:10.545200+0000","last_became_peered":"2024-09-21T02:09:10.545200+0000","last_unstale":"2024-09-21T02:09:16.726772+0000","last_undegraded":"2024-09-21T02:09:16.726772+0000","last_fullsized":"2024-09-21T02:09:16.726772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593400+0000","last_change":"2024-09-21T02:09:15.630472+0000","last_active":"2024-09-21T02:09:16.593400+0000","last_peered":"2024-09-21T02:09:16.593400+0000","last_clean":"2024-09-21T02:09:16.593400+0000","last_became_active":"2024-09-21T02:09:15.628764+0000","last_became_peered":"2024-09-21T02:09:15.628764+0000","last_unstale":"2024-09-21T02:09:16.593400+0000","last_undegraded":"2024-09-21T02:09:16.593400+0000","last_fullsized":"2024-09-21T02:09:16.593400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:21.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591110+0000","last_change":"2024-09-21T02:09:11.543344+0000","last_active":"2024-09-21T02:09:16.591110+0000","last_peered":"2024-09-21T02:09:16.591110+0000","last_clean":"2024-09-21T02:09:16.591110+0000","last_became_active":"2024-09-21T02:09:11.542450+0000","last_became_peered":"2024-09-21T02:09:11.542450+0000","last_unstale":"2024-09-21T02:09:16.591110+0000","last_undegraded":"2024-09-21T02:09:16.591110+0000","last_fullsized":"2024-09-21T02:09:16.591110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:49.206820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719673+0000","last_change":"2024-09-21T02:09:10.541105+0000","last_active":"2024-09-21T02:09:16.719673+0000","last_peered":"2024-09-21T02:09:16.719673+0000","last_clean":"2024-09-21T02:09:16.719673+0000","last_became_active":"2024-09-21T02:09:10.540744+0000","last_became_peered":"2024-09-21T02:09:10.540744+0000","last_unstale":"2024-09-21T02:09:16.719673+0000","last_undegraded":"2024-09-21T02:09:16.719673+0000","last_fullsized":"2024-09-21T02:09:16.719673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:02.568801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588758+0000","last_change":"2024-09-21T02:09:15.613004+0000","last_active":"2024-09-21T02:09:16.588758+0000","last_peered":"2024-09-21T02:09:16.588758+0000","last_clean":"2024-09-21T02:09:16.588758+0000","last_became_active":"2024-09-21T02:09:15.612265+0000","last_became_peered":"2024-09-21T02:09:15.612265+0000","last_unstale":"2024-09-21T02:09:16.588758+0000","last_undegraded":"2024-09-21T02:09:16.588758+0000","last_fullsized":"2024-09-21T02:09:16.588758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721841+0000","last_change":"2024-09-21T02:09:11.528642+0000","last_active":"2024-09-21T02:09:16.721841+0000","last_peered":"2024-09-21T02:09:16.721841+0000","last_clean":"2024-09-21T02:09:16.721841+0000","last_became_active":"2024-09-21T02:09:11.526472+0000","last_became_peered":"2024-09-21T02:09:11.526472+0000","last_unstale":"2024-09-21T02:09:16.721841+0000","last_undegraded":"2024-09-21T02:09:16.721841+0000","last_fullsized":"2024-09-21T02:09:16.721841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593724+0000","last_change":"2024-09-21T02:09:10.553157+0000","last_active":"2024-09-21T02:09:16.593724+0000","last_peered":"2024-09-21T02:09:16.593724+0000","last_clean":"2024-09-21T02:09:16.593724+0000","last_became_active":"2024-09-21T02:09:10.552668+0000","last_became_peered":"2024-09-21T02:09:10.552668+0000","last_unstale":"2024-09-21T02:09:16.593724+0000","last_undegraded":"2024-09-21T02:09:16.593724+0000","last_fullsized":"2024-09-21T02:09:16.593724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:02.667450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723243+0000","last_change":"2024-09-21T02:09:15.626993+0000","last_active":"2024-09-21T02:09:16.723243+0000","last_peered":"2024-09-21T02:09:16.723243+0000","last_clean":"2024-09-21T02:09:16.723243+0000","last_became_active":"2024-09-21T02:09:15.626367+0000","last_became_peered":"2024-09-21T02:09:15.626367+0000","last_unstale":"2024-09-21T02:09:16.723243+0000","last_undegraded":"2024-09-21T02:09:16.723243+0000","last_fullsized":"2024-09-21T02:09:16.723243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:31.903642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724350+0000","last_change":"2024-09-21T02:09:11.557042+0000","last_active":"2024-09-21T02:09:16.724350+0000","last_peered":"2024-09-21T02:09:16.724350+0000","last_clean":"2024-09-21T02:09:16.724350+0000","last_became_active":"2024-09-21T02:09:11.555201+0000","last_became_peered":"2024-09-21T02:09:11.555201+0000","last_unstale":"2024-09-21T02:09:16.724350+0000","last_undegraded":"2024-09-21T02:09:16.724350+0000","last_fullsized":"2024-09-21T02:09:16.724350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722129+0000","last_change":"2024-09-21T02:09:10.516313+0000","last_active":"2024-09-21T02:09:16.722129+0000","last_peered":"2024-09-21T02:09:16.722129+0000","last_clean":"2024-09-21T02:09:16.722129+0000","last_became_active":"2024-09-21T02:09:10.515863+0000","last_became_peered":"2024-09-21T02:09:10.515863+0000","last_unstale":"2024-09-21T02:09:16.722129+0000","last_undegraded":"2024-09-21T02:09:16.722129+0000","last_fullsized":"2024-09-21T02:09:16.722129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719183+0000","last_change":"2024-09-21T02:09:15.624503+0000","last_active":"2024-09-21T02:09:16.719183+0000","last_peered":"2024-09-21T02:09:16.719183+0000","last_clean":"2024-09-21T02:09:16.719183+0000","last_became_active":"2024-09-21T02:09:15.622674+0000","last_became_peered":"2024-09-21T02:09:15.622674+0000","last_unstale":"2024-09-21T02:09:16.719183+0000","last_undegraded":"2024-09-21T02:09:16.719183+0000","last_fullsized":"2024-09-21T02:09:16.719183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:32.890032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596034+0000","last_change":"2024-09-21T02:09:11.550090+0000","last_active":"2024-09-21T02:09:16.596034+0000","last_peered":"2024-09-21T02:09:16.596034+0000","last_clean":"2024-09-21T02:09:16.596034+0000","last_became_active":"2024-09-21T02:09:11.549246+0000","last_became_peered":"2024-09-21T02:09:11.549246+0000","last_unstale":"2024-09-21T02:09:16.596034+0000","last_undegraded":"2024-09-21T02:09:16.596034+0000","last_fullsized":"2024-09-21T02:09:16.596034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.842699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725146+0000","last_change":"2024-09-21T02:09:10.545849+0000","last_active":"2024-09-21T02:09:16.725146+0000","last_peered":"2024-09-21T02:09:16.725146+0000","last_clean":"2024-09-21T02:09:16.725146+0000","last_became_active":"2024-09-21T02:09:10.544961+0000","last_became_peered":"2024-09-21T02:09:10.544961+0000","last_unstale":"2024-09-21T02:09:16.725146+0000","last_undegraded":"2024-09-21T02:09:16.725146+0000","last_fullsized":"2024-09-21T02:09:16.725146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:43.073819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721335+0000","last_change":"2024-09-21T02:09:15.611136+0000","last_active":"2024-09-21T02:09:16.721335+0000","last_peered":"2024-09-21T02:09:16.721335+0000","last_clean":"2024-09-21T02:09:16.721335+0000","last_became_active":"2024-09-21T02:09:15.610710+0000","last_became_peered":"2024-09-21T02:09:15.610710+0000","last_unstale":"2024-09-21T02:09:16.721335+0000","last_undegraded":"2024-09-21T02:09:16.721335+0000","last_fullsized":"2024-09-21T02:09:16.721335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:11.070735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719495+0000","last_change":"2024-09-21T02:09:11.526818+0000","last_active":"2024-09-21T02:09:16.719495+0000","last_peered":"2024-09-21T02:09:16.719495+0000","last_clean":"2024-09-21T02:09:16.719495+0000","last_became_active":"2024-09-21T02:09:11.525899+0000","last_became_peered":"2024-09-21T02:09:11.525899+0000","last_unstale":"2024-09-21T02:09:16.719495+0000","last_undegraded":"2024-09-21T02:09:16.719495+0000","last_fullsized":"2024-09-21T02:09:16.719495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720218+0000","last_change":"2024-09-21T02:09:10.526158+0000","last_active":"2024-09-21T02:09:16.720218+0000","last_peered":"2024-09-21T02:09:16.720218+0000","last_clean":"2024-09-21T02:09:16.720218+0000","last_became_active":"2024-09-21T02:09:10.525593+0000","last_became_peered":"2024-09-21T02:09:10.525593+0000","last_unstale":"2024-09-21T02:09:16.720218+0000","last_undegraded":"2024-09-21T02:09:16.720218+0000","last_fullsized":"2024-09-21T02:09:16.720218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720063+0000","last_change":"2024-09-21T02:09:15.623852+0000","last_active":"2024-09-21T02:09:16.720063+0000","last_peered":"2024-09-21T02:09:16.720063+0000","last_clean":"2024-09-21T02:09:16.720063+0000","last_became_active":"2024-09-21T02:09:15.623080+0000","last_became_peered":"2024-09-21T02:09:15.623080+0000","last_unstale":"2024-09-21T02:09:16.720063+0000","last_undegraded":"2024-09-21T02:09:16.720063+0000","last_fullsized":"2024-09-21T02:09:16.720063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:38.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722226+0000","last_change":"2024-09-21T02:09:11.558074+0000","last_active":"2024-09-21T02:09:16.722226+0000","last_peered":"2024-09-21T02:09:16.722226+0000","last_clean":"2024-09-21T02:09:16.722226+0000","last_became_active":"2024-09-21T02:09:11.557270+0000","last_became_peered":"2024-09-21T02:09:11.557270+0000","last_unstale":"2024-09-21T02:09:16.722226+0000","last_undegraded":"2024-09-21T02:09:16.722226+0000","last_fullsized":"2024-09-21T02:09:16.722226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:29.728033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590847+0000","last_change":"2024-09-21T02:09:10.540346+0000","last_active":"2024-09-21T02:09:16.590847+0000","last_peered":"2024-09-21T02:09:16.590847+0000","last_clean":"2024-09-21T02:09:16.590847+0000","last_became_active":"2024-09-21T02:09:10.539632+0000","last_became_peered":"2024-09-21T02:09:10.539632+0000","last_unstale":"2024-09-21T02:09:16.590847+0000","last_undegraded":"2024-09-21T02:09:16.590847+0000","last_fullsized":"2024-09-21T02:09:16.590847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722704+0000","last_change":"2024-09-21T02:09:15.636027+0000","last_active":"2024-09-21T02:09:16.722704+0000","last_peered":"2024-09-21T02:09:16.722704+0000","last_clean":"2024-09-21T02:09:16.722704+0000","last_became_active":"2024-09-21T02:09:15.635644+0000","last_became_peered":"2024-09-21T02:09:15.635644+0000","last_unstale":"2024-09-21T02:09:16.722704+0000","last_undegraded":"2024-09-21T02:09:16.722704+0000","last_fullsized":"2024-09-21T02:09:16.722704+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:46.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589275+0000","last_change":"2024-09-21T02:09:11.568395+0000","last_active":"2024-09-21T02:09:16.589275+0000","last_peered":"2024-09-21T02:09:16.589275+0000","last_clean":"2024-09-21T02:09:16.589275+0000","last_became_active":"2024-09-21T02:09:11.567951+0000","last_became_peered":"2024-09-21T02:09:11.567951+0000","last_unstale":"2024-09-21T02:09:16.589275+0000","last_undegraded":"2024-09-21T02:09:16.589275+0000","last_fullsized":"2024-09-21T02:09:16.589275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:38.502107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724904+0000","last_change":"2024-09-21T02:09:10.542137+0000","last_active":"2024-09-21T02:09:16.724904+0000","last_peered":"2024-09-21T02:09:16.724904+0000","last_clean":"2024-09-21T02:09:16.724904+0000","last_became_active":"2024-09-21T02:09:10.541324+0000","last_became_peered":"2024-09-21T02:09:10.541324+0000","last_unstale":"2024-09-21T02:09:16.724904+0000","last_undegraded":"2024-09-21T02:09:16.724904+0000","last_fullsized":"2024-09-21T02:09:16.724904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591215+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:09:16.591215+0000","last_peered":"2024-09-21T02:09:16.591215+0000","last_clean":"2024-09-21T02:09:16.591215+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:09:16.591215+0000","last_undegraded":"2024-09-21T02:09:16.591215+0000","last_fullsized":"2024-09-21T02:09:16.591215+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725602+0000","last_change":"2024-09-21T02:09:15.613320+0000","last_active":"2024-09-21T02:09:16.725602+0000","last_peered":"2024-09-21T02:09:16.725602+0000","last_clean":"2024-09-21T02:09:16.725602+0000","last_became_active":"2024-09-21T02:09:15.612050+0000","last_became_peered":"2024-09-21T02:09:15.612050+0000","last_unstale":"2024-09-21T02:09:16.725602+0000","last_undegraded":"2024-09-21T02:09:16.725602+0000","last_fullsized":"2024-09-21T02:09:16.725602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:54.519499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592039+0000","last_change":"2024-09-21T02:09:11.571228+0000","last_active":"2024-09-21T02:09:16.592039+0000","last_peered":"2024-09-21T02:09:16.592039+0000","last_clean":"2024-09-21T02:09:16.592039+0000","last_became_active":"2024-09-21T02:09:11.570627+0000","last_became_peered":"2024-09-21T02:09:11.570627+0000","last_unstale":"2024-09-21T02:09:16.592039+0000","last_undegraded":"2024-09-21T02:09:16.592039+0000","last_fullsized":"2024-09-21T02:09:16.592039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.513856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721976+0000","last_change":"2024-09-21T02:09:10.530153+0000","last_active":"2024-09-21T02:09:16.721976+0000","last_peered":"2024-09-21T02:09:16.721976+0000","last_clean":"2024-09-21T02:09:16.721976+0000","last_became_active":"2024-09-21T02:09:10.529650+0000","last_became_peered":"2024-09-21T02:09:10.529650+0000","last_unstale":"2024-09-21T02:09:16.721976+0000","last_undegraded":"2024-09-21T02:09:16.721976+0000","last_fullsized":"2024-09-21T02:09:16.721976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724037+0000","last_change":"2024-09-21T02:09:15.609642+0000","last_active":"2024-09-21T02:09:16.724037+0000","last_peered":"2024-09-21T02:09:16.724037+0000","last_clean":"2024-09-21T02:09:16.724037+0000","last_became_active":"2024-09-21T02:09:15.604449+0000","last_became_peered":"2024-09-21T02:09:15.604449+0000","last_unstale":"2024-09-21T02:09:16.724037+0000","last_undegraded":"2024-09-21T02:09:16.724037+0000","last_fullsized":"2024-09-21T02:09:16.724037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:55.894914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724291+0000","last_change":"2024-09-21T02:09:11.554539+0000","last_active":"2024-09-21T02:09:16.724291+0000","last_peered":"2024-09-21T02:09:16.724291+0000","last_clean":"2024-09-21T02:09:16.724291+0000","last_became_active":"2024-09-21T02:09:11.553712+0000","last_became_peered":"2024-09-21T02:09:11.553712+0000","last_unstale":"2024-09-21T02:09:16.724291+0000","last_undegraded":"2024-09-21T02:09:16.724291+0000","last_fullsized":"2024-09-21T02:09:16.724291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.229624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719880+0000","last_change":"2024-09-21T02:09:10.545134+0000","last_active":"2024-09-21T02:09:16.719880+0000","last_peered":"2024-09-21T02:09:16.719880+0000","last_clean":"2024-09-21T02:09:16.719880+0000","last_became_active":"2024-09-21T02:09:10.544792+0000","last_became_peered":"2024-09-21T02:09:10.544792+0000","last_unstale":"2024-09-21T02:09:16.719880+0000","last_undegraded":"2024-09-21T02:09:16.719880+0000","last_fullsized":"2024-09-21T02:09:16.719880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723145+0000","last_change":"2024-09-21T02:09:15.626483+0000","last_active":"2024-09-21T02:09:16.723145+0000","last_peered":"2024-09-21T02:09:16.723145+0000","last_clean":"2024-09-21T02:09:16.723145+0000","last_became_active":"2024-09-21T02:09:15.626132+0000","last_became_peered":"2024-09-21T02:09:15.626132+0000","last_unstale":"2024-09-21T02:09:16.723145+0000","last_undegraded":"2024-09-21T02:09:16.723145+0000","last_fullsized":"2024-09-21T02:09:16.723145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:04.011318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592725+0000","last_change":"2024-09-21T02:09:11.533777+0000","last_active":"2024-09-21T02:09:16.592725+0000","last_peered":"2024-09-21T02:09:16.592725+0000","last_clean":"2024-09-21T02:09:16.592725+0000","last_became_active":"2024-09-21T02:09:11.532928+0000","last_became_peered":"2024-09-21T02:09:11.532928+0000","last_unstale":"2024-09-21T02:09:16.592725+0000","last_undegraded":"2024-09-21T02:09:16.592725+0000","last_fullsized":"2024-09-21T02:09:16.592725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:21.176822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721695+0000","last_change":"2024-09-21T02:09:10.524899+0000","last_active":"2024-09-21T02:09:16.721695+0000","last_peered":"2024-09-21T02:09:16.721695+0000","last_clean":"2024-09-21T02:09:16.721695+0000","last_became_active":"2024-09-21T02:09:10.524447+0000","last_became_peered":"2024-09-21T02:09:10.524447+0000","last_unstale":"2024-09-21T02:09:16.721695+0000","last_undegraded":"2024-09-21T02:09:16.721695+0000","last_fullsized":"2024-09-21T02:09:16.721695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723182+0000","last_change":"2024-09-21T02:09:15.623410+0000","last_active":"2024-09-21T02:09:16.723182+0000","last_peered":"2024-09-21T02:09:16.723182+0000","last_clean":"2024-09-21T02:09:16.723182+0000","last_became_active":"2024-09-21T02:09:15.622194+0000","last_became_peered":"2024-09-21T02:09:15.622194+0000","last_unstale":"2024-09-21T02:09:16.723182+0000","last_undegraded":"2024-09-21T02:09:16.723182+0000","last_fullsized":"2024-09-21T02:09:16.723182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594437+0000","last_change":"2024-09-21T02:09:11.531628+0000","last_active":"2024-09-21T02:09:16.594437+0000","last_peered":"2024-09-21T02:09:16.594437+0000","last_clean":"2024-09-21T02:09:16.594437+0000","last_became_active":"2024-09-21T02:09:11.522487+0000","last_became_peered":"2024-09-21T02:09:11.522487+0000","last_unstale":"2024-09-21T02:09:16.594437+0000","last_undegraded":"2024-09-21T02:09:16.594437+0000","last_fullsized":"2024-09-21T02:09:16.594437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:31.403053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595653+0000","last_change":"2024-09-21T02:09:10.526000+0000","last_active":"2024-09-21T02:09:16.595653+0000","last_peered":"2024-09-21T02:09:16.595653+0000","last_clean":"2024-09-21T02:09:16.595653+0000","last_became_active":"2024-09-21T02:09:10.525267+0000","last_became_peered":"2024-09-21T02:09:10.525267+0000","last_unstale":"2024-09-21T02:09:16.595653+0000","last_undegraded":"2024-09-21T02:09:16.595653+0000","last_fullsized":"2024-09-21T02:09:16.595653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:32.448144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592641+0000","last_change":"2024-09-21T02:09:15.626495+0000","last_active":"2024-09-21T02:09:16.592641+0000","last_peered":"2024-09-21T02:09:16.592641+0000","last_clean":"2024-09-21T02:09:16.592641+0000","last_became_active":"2024-09-21T02:09:15.625151+0000","last_became_peered":"2024-09-21T02:09:15.625151+0000","last_unstale":"2024-09-21T02:09:16.592641+0000","last_undegraded":"2024-09-21T02:09:16.592641+0000","last_fullsized":"2024-09-21T02:09:16.592641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:23.340983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591341+0000","last_change":"2024-09-21T02:09:11.521241+0000","last_active":"2024-09-21T02:09:16.591341+0000","last_peered":"2024-09-21T02:09:16.591341+0000","last_clean":"2024-09-21T02:09:16.591341+0000","last_became_active":"2024-09-21T02:09:11.519883+0000","last_became_peered":"2024-09-21T02:09:11.519883+0000","last_unstale":"2024-09-21T02:09:16.591341+0000","last_undegraded":"2024-09-21T02:09:16.591341+0000","last_fullsized":"2024-09-21T02:09:16.591341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:26.260780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720026+0000","last_change":"2024-09-21T02:09:10.520638+0000","last_active":"2024-09-21T02:09:16.720026+0000","last_peered":"2024-09-21T02:09:16.720026+0000","last_clean":"2024-09-21T02:09:16.720026+0000","last_became_active":"2024-09-21T02:09:10.520131+0000","last_became_peered":"2024-09-21T02:09:10.520131+0000","last_unstale":"2024-09-21T02:09:16.720026+0000","last_undegraded":"2024-09-21T02:09:16.720026+0000","last_fullsized":"2024-09-21T02:09:16.720026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723034+0000","last_change":"2024-09-21T02:09:15.610523+0000","last_active":"2024-09-21T02:09:16.723034+0000","last_peered":"2024-09-21T02:09:16.723034+0000","last_clean":"2024-09-21T02:09:16.723034+0000","last_became_active":"2024-09-21T02:09:15.609441+0000","last_became_peered":"2024-09-21T02:09:15.609441+0000","last_unstale":"2024-09-21T02:09:16.723034+0000","last_undegraded":"2024-09-21T02:09:16.723034+0000","last_fullsized":"2024-09-21T02:09:16.723034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:36.011939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722864+0000","last_change":"2024-09-21T02:09:11.561816+0000","last_active":"2024-09-21T02:09:16.722864+0000","last_peered":"2024-09-21T02:09:16.722864+0000","last_clean":"2024-09-21T02:09:16.722864+0000","last_became_active":"2024-09-21T02:09:11.561231+0000","last_became_peered":"2024-09-21T02:09:11.561231+0000","last_unstale":"2024-09-21T02:09:16.722864+0000","last_undegraded":"2024-09-21T02:09:16.722864+0000","last_fullsized":"2024-09-21T02:09:16.722864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:38.366733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723173+0000","last_change":"2024-09-21T02:09:10.550508+0000","last_active":"2024-09-21T02:09:16.723173+0000","last_peered":"2024-09-21T02:09:16.723173+0000","last_clean":"2024-09-21T02:09:16.723173+0000","last_became_active":"2024-09-21T02:09:10.549913+0000","last_became_peered":"2024-09-21T02:09:10.549913+0000","last_unstale":"2024-09-21T02:09:16.723173+0000","last_undegraded":"2024-09-21T02:09:16.723173+0000","last_fullsized":"2024-09-21T02:09:16.723173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:15.201315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591175+0000","last_change":"2024-09-21T02:09:15.618146+0000","last_active":"2024-09-21T02:09:16.591175+0000","last_peered":"2024-09-21T02:09:16.591175+0000","last_clean":"2024-09-21T02:09:16.591175+0000","last_became_active":"2024-09-21T02:09:15.617195+0000","last_became_peered":"2024-09-21T02:09:15.617195+0000","last_unstale":"2024-09-21T02:09:16.591175+0000","last_undegraded":"2024-09-21T02:09:16.591175+0000","last_fullsized":"2024-09-21T02:09:16.591175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.452372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592327+0000","last_change":"2024-09-21T02:09:11.520992+0000","last_active":"2024-09-21T02:09:16.592327+0000","last_peered":"2024-09-21T02:09:16.592327+0000","last_clean":"2024-09-21T02:09:16.592327+0000","last_became_active":"2024-09-21T02:09:11.519892+0000","last_became_peered":"2024-09-21T02:09:11.519892+0000","last_unstale":"2024-09-21T02:09:16.592327+0000","last_undegraded":"2024-09-21T02:09:16.592327+0000","last_fullsized":"2024-09-21T02:09:16.592327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:34.673193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724869+0000","last_change":"2024-09-21T02:09:10.543820+0000","last_active":"2024-09-21T02:09:16.724869+0000","last_peered":"2024-09-21T02:09:16.724869+0000","last_clean":"2024-09-21T02:09:16.724869+0000","last_became_active":"2024-09-21T02:09:10.542944+0000","last_became_peered":"2024-09-21T02:09:10.542944+0000","last_unstale":"2024-09-21T02:09:16.724869+0000","last_undegraded":"2024-09-21T02:09:16.724869+0000","last_fullsized":"2024-09-21T02:09:16.724869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:59.409264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590058+0000","last_change":"2024-09-21T02:09:15.612341+0000","last_active":"2024-09-21T02:09:16.590058+0000","last_peered":"2024-09-21T02:09:16.590058+0000","last_clean":"2024-09-21T02:09:16.590058+0000","last_became_active":"2024-09-21T02:09:15.611864+0000","last_became_peered":"2024-09-21T02:09:15.611864+0000","last_unstale":"2024-09-21T02:09:16.590058+0000","last_undegraded":"2024-09-21T02:09:16.590058+0000","last_fullsized":"2024-09-21T02:09:16.590058+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:25.484143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723926+0000","last_change":"2024-09-21T02:09:11.557442+0000","last_active":"2024-09-21T02:09:16.723926+0000","last_peered":"2024-09-21T02:09:16.723926+0000","last_clean":"2024-09-21T02:09:16.723926+0000","last_became_active":"2024-09-21T02:09:11.557119+0000","last_became_peered":"2024-09-21T02:09:11.557119+0000","last_unstale":"2024-09-21T02:09:16.723926+0000","last_undegraded":"2024-09-21T02:09:16.723926+0000","last_fullsized":"2024-09-21T02:09:16.723926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723410+0000","last_change":"2024-09-21T02:09:10.530343+0000","last_active":"2024-09-21T02:09:16.723410+0000","last_peered":"2024-09-21T02:09:16.723410+0000","last_clean":"2024-09-21T02:09:16.723410+0000","last_became_active":"2024-09-21T02:09:10.529930+0000","last_became_peered":"2024-09-21T02:09:10.529930+0000","last_unstale":"2024-09-21T02:09:16.723410+0000","last_undegraded":"2024-09-21T02:09:16.723410+0000","last_fullsized":"2024-09-21T02:09:16.723410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725837+0000","last_change":"2024-09-21T02:09:15.635216+0000","last_active":"2024-09-21T02:09:16.725837+0000","last_peered":"2024-09-21T02:09:16.725837+0000","last_clean":"2024-09-21T02:09:16.725837+0000","last_became_active":"2024-09-21T02:09:15.626937+0000","last_became_peered":"2024-09-21T02:09:15.626937+0000","last_unstale":"2024-09-21T02:09:16.725837+0000","last_undegraded":"2024-09-21T02:09:16.725837+0000","last_fullsized":"2024-09-21T02:09:16.725837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591747+0000","last_change":"2024-09-21T02:09:11.538575+0000","last_active":"2024-09-21T02:09:16.591747+0000","last_peered":"2024-09-21T02:09:16.591747+0000","last_clean":"2024-09-21T02:09:16.591747+0000","last_became_active":"2024-09-21T02:09:11.535561+0000","last_became_peered":"2024-09-21T02:09:11.535561+0000","last_unstale":"2024-09-21T02:09:16.591747+0000","last_undegraded":"2024-09-21T02:09:16.591747+0000","last_fullsized":"2024-09-21T02:09:16.591747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:58.627736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:22.534320+0000","last_change":"2024-09-21T02:09:10.546712+0000","last_active":"2024-09-21T02:09:22.534320+0000","last_peered":"2024-09-21T02:09:22.534320+0000","last_clean":"2024-09-21T02:09:22.534320+0000","last_became_active":"2024-09-21T02:09:10.541968+0000","last_became_peered":"2024-09-21T02:09:10.541968+0000","last_unstale":"2024-09-21T02:09:22.534320+0000","last_undegraded":"2024-09-21T02:09:22.534320+0000","last_fullsized":"2024-09-21T02:09:22.534320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720210+0000","last_change":"2024-09-21T02:09:15.616116+0000","last_active":"2024-09-21T02:09:16.720210+0000","last_peered":"2024-09-21T02:09:16.720210+0000","last_clean":"2024-09-21T02:09:16.720210+0000","last_became_active":"2024-09-21T02:09:15.615337+0000","last_became_peered":"2024-09-21T02:09:15.615337+0000","last_unstale":"2024-09-21T02:09:16.720210+0000","last_undegraded":"2024-09-21T02:09:16.720210+0000","last_fullsized":"2024-09-21T02:09:16.720210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:35.472712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594788+0000","last_change":"2024-09-21T02:09:11.520271+0000","last_active":"2024-09-21T02:09:16.594788+0000","last_peered":"2024-09-21T02:09:16.594788+0000","last_clean":"2024-09-21T02:09:16.594788+0000","last_became_active":"2024-09-21T02:09:11.518967+0000","last_became_peered":"2024-09-21T02:09:11.518967+0000","last_unstale":"2024-09-21T02:09:16.594788+0000","last_undegraded":"2024-09-21T02:09:16.594788+0000","last_fullsized":"2024-09-21T02:09:16.594788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719861+0000","last_change":"2024-09-21T02:09:10.536568+0000","last_active":"2024-09-21T02:09:16.719861+0000","last_peered":"2024-09-21T02:09:16.719861+0000","last_clean":"2024-09-21T02:09:16.719861+0000","last_became_active":"2024-09-21T02:09:10.536266+0000","last_became_peered":"2024-09-21T02:09:10.536266+0000","last_unstale":"2024-09-21T02:09:16.719861+0000","last_undegraded":"2024-09-21T02:09:16.719861+0000","last_fullsized":"2024-09-21T02:09:16.719861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595233+0000","last_change":"2024-09-21T02:09:15.637262+0000","last_active":"2024-09-21T02:09:16.595233+0000","last_peered":"2024-09-21T02:09:16.595233+0000","last_clean":"2024-09-21T02:09:16.595233+0000","last_became_active":"2024-09-21T02:09:15.636305+0000","last_became_peered":"2024-09-21T02:09:15.636305+0000","last_unstale":"2024-09-21T02:09:16.595233+0000","last_undegraded":"2024-09-21T02:09:16.595233+0000","last_fullsized":"2024-09-21T02:09:16.595233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:36.447874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723352+0000","last_change":"2024-09-21T02:09:11.562979+0000","last_active":"2024-09-21T02:09:16.723352+0000","last_peered":"2024-09-21T02:09:16.723352+0000","last_clean":"2024-09-21T02:09:16.723352+0000","last_became_active":"2024-09-21T02:09:11.559009+0000","last_became_peered":"2024-09-21T02:09:11.559009+0000","last_unstale":"2024-09-21T02:09:16.723352+0000","last_undegraded":"2024-09-21T02:09:16.723352+0000","last_fullsized":"2024-09-21T02:09:16.723352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:27.296832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725417+0000","last_change":"2024-09-21T02:09:10.546521+0000","last_active":"2024-09-21T02:09:16.725417+0000","last_peered":"2024-09-21T02:09:16.725417+0000","last_clean":"2024-09-21T02:09:16.725417+0000","last_became_active":"2024-09-21T02:09:10.545864+0000","last_became_peered":"2024-09-21T02:09:10.545864+0000","last_unstale":"2024-09-21T02:09:16.725417+0000","last_undegraded":"2024-09-21T02:09:16.725417+0000","last_fullsized":"2024-09-21T02:09:16.725417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589335+0000","last_change":"2024-09-21T02:09:15.616179+0000","last_active":"2024-09-21T02:09:16.589335+0000","last_peered":"2024-09-21T02:09:16.589335+0000","last_clean":"2024-09-21T02:09:16.589335+0000","last_became_active":"2024-09-21T02:09:15.615344+0000","last_became_peered":"2024-09-21T02:09:15.615344+0000","last_unstale":"2024-09-21T02:09:16.589335+0000","last_undegraded":"2024-09-21T02:09:16.589335+0000","last_fullsized":"2024-09-21T02:09:16.589335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.519680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725123+0000","last_change":"2024-09-21T02:09:11.582832+0000","last_active":"2024-09-21T02:09:16.725123+0000","last_peered":"2024-09-21T02:09:16.725123+0000","last_clean":"2024-09-21T02:09:16.725123+0000","last_became_active":"2024-09-21T02:09:11.582121+0000","last_became_peered":"2024-09-21T02:09:11.582121+0000","last_unstale":"2024-09-21T02:09:16.725123+0000","last_undegraded":"2024-09-21T02:09:16.725123+0000","last_fullsized":"2024-09-21T02:09:16.725123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.829896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726463+0000","last_change":"2024-09-21T02:09:10.556671+0000","last_active":"2024-09-21T02:09:16.726463+0000","last_peered":"2024-09-21T02:09:16.726463+0000","last_clean":"2024-09-21T02:09:16.726463+0000","last_became_active":"2024-09-21T02:09:10.555875+0000","last_became_peered":"2024-09-21T02:09:10.555875+0000","last_unstale":"2024-09-21T02:09:16.726463+0000","last_undegraded":"2024-09-21T02:09:16.726463+0000","last_fullsized":"2024-09-21T02:09:16.726463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588965+0000","last_change":"2024-09-21T02:09:15.623197+0000","last_active":"2024-09-21T02:09:16.588965+0000","last_peered":"2024-09-21T02:09:16.588965+0000","last_clean":"2024-09-21T02:09:16.588965+0000","last_became_active":"2024-09-21T02:09:15.622340+0000","last_became_peered":"2024-09-21T02:09:15.622340+0000","last_unstale":"2024-09-21T02:09:16.588965+0000","last_undegraded":"2024-09-21T02:09:16.588965+0000","last_fullsized":"2024-09-21T02:09:16.588965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722347+0000","last_change":"2024-09-21T02:09:15.636227+0000","last_active":"2024-09-21T02:09:16.722347+0000","last_peered":"2024-09-21T02:09:16.722347+0000","last_clean":"2024-09-21T02:09:16.722347+0000","last_became_active":"2024-09-21T02:09:15.635814+0000","last_became_peered":"2024-09-21T02:09:15.635814+0000","last_unstale":"2024-09-21T02:09:16.722347+0000","last_undegraded":"2024-09-21T02:09:16.722347+0000","last_fullsized":"2024-09-21T02:09:16.722347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:37.103933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719830+0000","last_change":"2024-09-21T02:09:11.531143+0000","last_active":"2024-09-21T02:09:16.719830+0000","last_peered":"2024-09-21T02:09:16.719830+0000","last_clean":"2024-09-21T02:09:16.719830+0000","last_became_active":"2024-09-21T02:09:11.530348+0000","last_became_peered":"2024-09-21T02:09:11.530348+0000","last_unstale":"2024-09-21T02:09:16.719830+0000","last_undegraded":"2024-09-21T02:09:16.719830+0000","last_fullsized":"2024-09-21T02:09:16.719830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:21.823287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723931+0000","last_change":"2024-09-21T02:09:10.538525+0000","last_active":"2024-09-21T02:09:16.723931+0000","last_peered":"2024-09-21T02:09:16.723931+0000","last_clean":"2024-09-21T02:09:16.723931+0000","last_became_active":"2024-09-21T02:09:10.537466+0000","last_became_peered":"2024-09-21T02:09:10.537466+0000","last_unstale":"2024-09-21T02:09:16.723931+0000","last_undegraded":"2024-09-21T02:09:16.723931+0000","last_fullsized":"2024-09-21T02:09:16.723931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726815+0000","last_change":"2024-09-21T02:09:15.625976+0000","last_active":"2024-09-21T02:09:16.726815+0000","last_peered":"2024-09-21T02:09:16.726815+0000","last_clean":"2024-09-21T02:09:16.726815+0000","last_became_active":"2024-09-21T02:09:15.624692+0000","last_became_peered":"2024-09-21T02:09:15.624692+0000","last_unstale":"2024-09-21T02:09:16.726815+0000","last_undegraded":"2024-09-21T02:09:16.726815+0000","last_fullsized":"2024-09-21T02:09:16.726815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:18.311891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588890+0000","last_change":"2024-09-21T02:09:11.544871+0000","last_active":"2024-09-21T02:09:16.588890+0000","last_peered":"2024-09-21T02:09:16.588890+0000","last_clean":"2024-09-21T02:09:16.588890+0000","last_became_active":"2024-09-21T02:09:11.543403+0000","last_became_peered":"2024-09-21T02:09:11.543403+0000","last_unstale":"2024-09-21T02:09:16.588890+0000","last_undegraded":"2024-09-21T02:09:16.588890+0000","last_fullsized":"2024-09-21T02:09:16.588890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589183+0000","last_change":"2024-09-21T02:09:10.539555+0000","last_active":"2024-09-21T02:09:16.589183+0000","last_peered":"2024-09-21T02:09:16.589183+0000","last_clean":"2024-09-21T02:09:16.589183+0000","last_became_active":"2024-09-21T02:09:10.539163+0000","last_became_peered":"2024-09-21T02:09:10.539163+0000","last_unstale":"2024-09-21T02:09:16.589183+0000","last_undegraded":"2024-09-21T02:09:16.589183+0000","last_fullsized":"2024-09-21T02:09:16.589183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724509+0000","last_change":"2024-09-21T02:09:10.550879+0000","last_active":"2024-09-21T02:09:16.724509+0000","last_peered":"2024-09-21T02:09:16.724509+0000","last_clean":"2024-09-21T02:09:16.724509+0000","last_became_active":"2024-09-21T02:09:10.550147+0000","last_became_peered":"2024-09-21T02:09:10.550147+0000","last_unstale":"2024-09-21T02:09:16.724509+0000","last_undegraded":"2024-09-21T02:09:16.724509+0000","last_fullsized":"2024-09-21T02:09:16.724509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723322+0000","last_change":"2024-09-21T02:09:11.564520+0000","last_active":"2024-09-21T02:09:16.723322+0000","last_peered":"2024-09-21T02:09:16.723322+0000","last_clean":"2024-09-21T02:09:16.723322+0000","last_became_active":"2024-09-21T02:09:11.563870+0000","last_became_peered":"2024-09-21T02:09:11.563870+0000","last_unstale":"2024-09-21T02:09:16.723322+0000","last_undegraded":"2024-09-21T02:09:16.723322+0000","last_fullsized":"2024-09-21T02:09:16.723322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:20.175775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721646+0000","last_change":"2024-09-21T02:09:15.623267+0000","last_active":"2024-09-21T02:09:16.721646+0000","last_peered":"2024-09-21T02:09:16.721646+0000","last_clean":"2024-09-21T02:09:16.721646+0000","last_became_active":"2024-09-21T02:09:15.622816+0000","last_became_peered":"2024-09-21T02:09:15.622816+0000","last_unstale":"2024-09-21T02:09:16.721646+0000","last_undegraded":"2024-09-21T02:09:16.721646+0000","last_fullsized":"2024-09-21T02:09:16.721646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:11.398708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589624+0000","last_change":"2024-09-21T02:09:10.545988+0000","last_active":"2024-09-21T02:09:16.589624+0000","last_peered":"2024-09-21T02:09:16.589624+0000","last_clean":"2024-09-21T02:09:16.589624+0000","last_became_active":"2024-09-21T02:09:10.545537+0000","last_became_peered":"2024-09-21T02:09:10.545537+0000","last_unstale":"2024-09-21T02:09:16.589624+0000","last_undegraded":"2024-09-21T02:09:16.589624+0000","last_fullsized":"2024-09-21T02:09:16.589624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592924+0000","last_change":"2024-09-21T02:09:11.527921+0000","last_active":"2024-09-21T02:09:16.592924+0000","last_peered":"2024-09-21T02:09:16.592924+0000","last_clean":"2024-09-21T02:09:16.592924+0000","last_became_active":"2024-09-21T02:09:11.527033+0000","last_became_peered":"2024-09-21T02:09:11.527033+0000","last_unstale":"2024-09-21T02:09:16.592924+0000","last_undegraded":"2024-09-21T02:09:16.592924+0000","last_fullsized":"2024-09-21T02:09:16.592924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590282+0000","last_change":"2024-09-21T02:09:15.608837+0000","last_active":"2024-09-21T02:09:16.590282+0000","last_peered":"2024-09-21T02:09:16.590282+0000","last_clean":"2024-09-21T02:09:16.590282+0000","last_became_active":"2024-09-21T02:09:15.607905+0000","last_became_peered":"2024-09-21T02:09:15.607905+0000","last_unstale":"2024-09-21T02:09:16.590282+0000","last_undegraded":"2024-09-21T02:09:16.590282+0000","last_fullsized":"2024-09-21T02:09:16.590282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:15.347636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724603+0000","last_change":"2024-09-21T02:09:11.562756+0000","last_active":"2024-09-21T02:09:16.724603+0000","last_peered":"2024-09-21T02:09:16.724603+0000","last_clean":"2024-09-21T02:09:16.724603+0000","last_became_active":"2024-09-21T02:09:11.560604+0000","last_became_peered":"2024-09-21T02:09:11.560604+0000","last_unstale":"2024-09-21T02:09:16.724603+0000","last_undegraded":"2024-09-21T02:09:16.724603+0000","last_fullsized":"2024-09-21T02:09:16.724603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719949+0000","last_change":"2024-09-21T02:09:11.566770+0000","last_active":"2024-09-21T02:09:16.719949+0000","last_peered":"2024-09-21T02:09:16.719949+0000","last_clean":"2024-09-21T02:09:16.719949+0000","last_became_active":"2024-09-21T02:09:11.566425+0000","last_became_peered":"2024-09-21T02:09:11.566425+0000","last_unstale":"2024-09-21T02:09:16.719949+0000","last_undegraded":"2024-09-21T02:09:16.719949+0000","last_fullsized":"2024-09-21T02:09:16.719949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724847+0000","last_change":"2024-09-21T02:09:11.572089+0000","last_active":"2024-09-21T02:09:16.724847+0000","last_peered":"2024-09-21T02:09:16.724847+0000","last_clean":"2024-09-21T02:09:16.724847+0000","last_became_active":"2024-09-21T02:09:11.566576+0000","last_became_peered":"2024-09-21T02:09:11.566576+0000","last_unstale":"2024-09-21T02:09:16.724847+0000","last_undegraded":"2024-09-21T02:09:16.724847+0000","last_fullsized":"2024-09-21T02:09:16.724847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:42.706829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723234+0000","last_change":"2024-09-21T02:09:11.564192+0000","last_active":"2024-09-21T02:09:16.723234+0000","last_peered":"2024-09-21T02:09:16.723234+0000","last_clean":"2024-09-21T02:09:16.723234+0000","last_became_active":"2024-09-21T02:09:11.559079+0000","last_became_peered":"2024-09-21T02:09:11.559079+0000","last_unstale":"2024-09-21T02:09:16.723234+0000","last_undegraded":"2024-09-21T02:09:16.723234+0000","last_fullsized":"2024-09-21T02:09:16.723234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590380+0000","last_change":"2024-09-21T02:09:11.540588+0000","last_active":"2024-09-21T02:09:16.590380+0000","last_peered":"2024-09-21T02:09:16.590380+0000","last_clean":"2024-09-21T02:09:16.590380+0000","last_became_active":"2024-09-21T02:09:11.539779+0000","last_became_peered":"2024-09-21T02:09:11.539779+0000","last_unstale":"2024-09-21T02:09:16.590380+0000","last_undegraded":"2024-09-21T02:09:16.590380+0000","last_fullsized":"2024-09-21T02:09:16.590380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:24.826345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719412+0000","last_change":"2024-09-21T02:09:11.530898+0000","last_active":"2024-09-21T02:09:16.719412+0000","last_peered":"2024-09-21T02:09:16.719412+0000","last_clean":"2024-09-21T02:09:16.719412+0000","last_became_active":"2024-09-21T02:09:11.530100+0000","last_became_peered":"2024-09-21T02:09:11.530100+0000","last_unstale":"2024-09-21T02:09:16.719412+0000","last_undegraded":"2024-09-21T02:09:16.719412+0000","last_fullsized":"2024-09-21T02:09:16.719412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726838+0000","last_change":"2024-09-21T02:09:11.549198+0000","last_active":"2024-09-21T02:09:16.726838+0000","last_peered":"2024-09-21T02:09:16.726838+0000","last_clean":"2024-09-21T02:09:16.726838+0000","last_became_active":"2024-09-21T02:09:11.548421+0000","last_became_peered":"2024-09-21T02:09:11.548421+0000","last_unstale":"2024-09-21T02:09:16.726838+0000","last_undegraded":"2024-09-21T02:09:16.726838+0000","last_fullsized":"2024-09-21T02:09:16.726838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:22.505823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590900+0000","last_change":"2024-09-21T02:09:11.552426+0000","last_active":"2024-09-21T02:09:16.590900+0000","last_peered":"2024-09-21T02:09:16.590900+0000","last_clean":"2024-09-21T02:09:16.590900+0000","last_became_active":"2024-09-21T02:09:11.551844+0000","last_became_peered":"2024-09-21T02:09:11.551844+0000","last_unstale":"2024-09-21T02:09:16.590900+0000","last_undegraded":"2024-09-21T02:09:16.590900+0000","last_fullsized":"2024-09-21T02:09:16.590900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723635+0000","last_change":"2024-09-21T02:09:11.566549+0000","last_active":"2024-09-21T02:09:16.723635+0000","last_peered":"2024-09-21T02:09:16.723635+0000","last_clean":"2024-09-21T02:09:16.723635+0000","last_became_active":"2024-09-21T02:09:11.565905+0000","last_became_peered":"2024-09-21T02:09:11.565905+0000","last_unstale":"2024-09-21T02:09:16.723635+0000","last_undegraded":"2024-09-21T02:09:16.723635+0000","last_fullsized":"2024-09-21T02:09:16.723635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.460538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589185+0000","last_change":"2024-09-21T02:09:11.552230+0000","last_active":"2024-09-21T02:09:16.589185+0000","last_peered":"2024-09-21T02:09:16.589185+0000","last_clean":"2024-09-21T02:09:16.589185+0000","last_became_active":"2024-09-21T02:09:11.551873+0000","last_became_peered":"2024-09-21T02:09:11.551873+0000","last_unstale":"2024-09-21T02:09:16.589185+0000","last_undegraded":"2024-09-21T02:09:16.589185+0000","last_fullsized":"2024-09-21T02:09:16.589185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724470+0000","last_change":"2024-09-21T02:09:11.581809+0000","last_active":"2024-09-21T02:09:16.724470+0000","last_peered":"2024-09-21T02:09:16.724470+0000","last_clean":"2024-09-21T02:09:16.724470+0000","last_became_active":"2024-09-21T02:09:11.581010+0000","last_became_peered":"2024-09-21T02:09:11.581010+0000","last_unstale":"2024-09-21T02:09:16.724470+0000","last_undegraded":"2024-09-21T02:09:16.724470+0000","last_fullsized":"2024-09-21T02:09:16.724470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:19.271324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722807+0000","last_change":"2024-09-21T02:09:11.558586+0000","last_active":"2024-09-21T02:09:16.722807+0000","last_peered":"2024-09-21T02:09:16.722807+0000","last_clean":"2024-09-21T02:09:16.722807+0000","last_became_active":"2024-09-21T02:09:11.558126+0000","last_became_peered":"2024-09-21T02:09:11.558126+0000","last_unstale":"2024-09-21T02:09:16.722807+0000","last_undegraded":"2024-09-21T02:09:16.722807+0000","last_fullsized":"2024-09-21T02:09:16.722807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:10.080231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725755+0000","last_change":"2024-09-21T02:09:11.561183+0000","last_active":"2024-09-21T02:09:16.725755+0000","last_peered":"2024-09-21T02:09:16.725755+0000","last_clean":"2024-09-21T02:09:16.725755+0000","last_became_active":"2024-09-21T02:09:11.560260+0000","last_became_peered":"2024-09-21T02:09:11.560260+0000","last_unstale":"2024-09-21T02:09:16.725755+0000","last_undegraded":"2024-09-21T02:09:16.725755+0000","last_fullsized":"2024-09-21T02:09:16.725755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593415+0000","last_change":"2024-09-21T02:09:11.540272+0000","last_active":"2024-09-21T02:09:16.593415+0000","last_peered":"2024-09-21T02:09:16.593415+0000","last_clean":"2024-09-21T02:09:16.593415+0000","last_became_active":"2024-09-21T02:09:11.539409+0000","last_became_peered":"2024-09-21T02:09:11.539409+0000","last_unstale":"2024-09-21T02:09:16.593415+0000","last_undegraded":"2024-09-21T02:09:16.593415+0000","last_fullsized":"2024-09-21T02:09:16.593415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:28.820372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721399+0000","last_change":"2024-09-21T02:09:11.544995+0000","last_active":"2024-09-21T02:09:16.721399+0000","last_peered":"2024-09-21T02:09:16.721399+0000","last_clean":"2024-09-21T02:09:16.721399+0000","last_became_active":"2024-09-21T02:09:11.544306+0000","last_became_peered":"2024-09-21T02:09:11.544306+0000","last_unstale":"2024-09-21T02:09:16.721399+0000","last_undegraded":"2024-09-21T02:09:16.721399+0000","last_fullsized":"2024-09-21T02:09:16.721399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725507+0000","last_change":"2024-09-21T02:09:11.582210+0000","last_active":"2024-09-21T02:09:16.725507+0000","last_peered":"2024-09-21T02:09:16.725507+0000","last_clean":"2024-09-21T02:09:16.725507+0000","last_became_active":"2024-09-21T02:09:11.581734+0000","last_became_peered":"2024-09-21T02:09:11.581734+0000","last_unstale":"2024-09-21T02:09:16.725507+0000","last_undegraded":"2024-09-21T02:09:16.725507+0000","last_fullsized":"2024-09-21T02:09:16.725507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:01.926437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723576+0000","last_change":"2024-09-21T02:09:11.574300+0000","last_active":"2024-09-21T02:09:16.723576+0000","last_peered":"2024-09-21T02:09:16.723576+0000","last_clean":"2024-09-21T02:09:16.723576+0000","last_became_active":"2024-09-21T02:09:11.573443+0000","last_became_peered":"2024-09-21T02:09:11.573443+0000","last_unstale":"2024-09-21T02:09:16.723576+0000","last_undegraded":"2024-09-21T02:09:16.723576+0000","last_fullsized":"2024-09-21T02:09:16.723576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720120+0000","last_change":"2024-09-21T02:09:11.518238+0000","last_active":"2024-09-21T02:09:16.720120+0000","last_peered":"2024-09-21T02:09:16.720120+0000","last_clean":"2024-09-21T02:09:16.720120+0000","last_became_active":"2024-09-21T02:09:11.517379+0000","last_became_peered":"2024-09-21T02:09:11.517379+0000","last_unstale":"2024-09-21T02:09:16.720120+0000","last_undegraded":"2024-09-21T02:09:16.720120+0000","last_fullsized":"2024-09-21T02:09:16.720120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723824+0000","last_change":"2024-09-21T02:09:11.536912+0000","last_active":"2024-09-21T02:09:16.723824+0000","last_peered":"2024-09-21T02:09:16.723824+0000","last_clean":"2024-09-21T02:09:16.723824+0000","last_became_active":"2024-09-21T02:09:11.535607+0000","last_became_peered":"2024-09-21T02:09:11.535607+0000","last_unstale":"2024-09-21T02:09:16.723824+0000","last_undegraded":"2024-09-21T02:09:16.723824+0000","last_fullsized":"2024-09-21T02:09:16.723824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589439+0000","last_change":"2024-09-21T02:09:11.546912+0000","last_active":"2024-09-21T02:09:16.589439+0000","last_peered":"2024-09-21T02:09:16.589439+0000","last_clean":"2024-09-21T02:09:16.589439+0000","last_became_active":"2024-09-21T02:09:11.543438+0000","last_became_peered":"2024-09-21T02:09:11.543438+0000","last_unstale":"2024-09-21T02:09:16.589439+0000","last_undegraded":"2024-09-21T02:09:16.589439+0000","last_fullsized":"2024-09-21T02:09:16.589439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595165+0000","last_change":"2024-09-21T02:09:11.547068+0000","last_active":"2024-09-21T02:09:16.595165+0000","last_peered":"2024-09-21T02:09:16.595165+0000","last_clean":"2024-09-21T02:09:16.595165+0000","last_became_active":"2024-09-21T02:09:11.545661+0000","last_became_peered":"2024-09-21T02:09:11.545661+0000","last_unstale":"2024-09-21T02:09:16.595165+0000","last_undegraded":"2024-09-21T02:09:16.595165+0000","last_fullsized":"2024-09-21T02:09:16.595165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:00.421056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725978+0000","last_change":"2024-09-21T02:09:11.571193+0000","last_active":"2024-09-21T02:09:16.725978+0000","last_peered":"2024-09-21T02:09:16.725978+0000","last_clean":"2024-09-21T02:09:16.725978+0000","last_became_active":"2024-09-21T02:09:11.563913+0000","last_became_peered":"2024-09-21T02:09:11.563913+0000","last_unstale":"2024-09-21T02:09:16.725978+0000","last_undegraded":"2024-09-21T02:09:16.725978+0000","last_fullsized":"2024-09-21T02:09:16.725978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726661+0000","last_change":"2024-09-21T02:09:11.564018+0000","last_active":"2024-09-21T02:09:16.726661+0000","last_peered":"2024-09-21T02:09:16.726661+0000","last_clean":"2024-09-21T02:09:16.726661+0000","last_became_active":"2024-09-21T02:09:11.563374+0000","last_became_peered":"2024-09-21T02:09:11.563374+0000","last_unstale":"2024-09-21T02:09:16.726661+0000","last_undegraded":"2024-09-21T02:09:16.726661+0000","last_fullsized":"2024-09-21T02:09:16.726661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592542+0000","last_change":"2024-09-21T02:09:11.571561+0000","last_active":"2024-09-21T02:09:16.592542+0000","last_peered":"2024-09-21T02:09:16.592542+0000","last_clean":"2024-09-21T02:09:16.592542+0000","last_became_active":"2024-09-21T02:09:11.570942+0000","last_became_peered":"2024-09-21T02:09:11.570942+0000","last_unstale":"2024-09-21T02:09:16.592542+0000","last_undegraded":"2024-09-21T02:09:16.592542+0000","last_fullsized":"2024-09-21T02:09:16.592542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:07.001076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588463+0000","last_change":"2024-09-21T02:09:11.568051+0000","last_active":"2024-09-21T02:09:16.588463+0000","last_peered":"2024-09-21T02:09:16.588463+0000","last_clean":"2024-09-21T02:09:16.588463+0000","last_became_active":"2024-09-21T02:09:11.567478+0000","last_became_peered":"2024-09-21T02:09:11.567478+0000","last_unstale":"2024-09-21T02:09:16.588463+0000","last_undegraded":"2024-09-21T02:09:16.588463+0000","last_fullsized":"2024-09-21T02:09:16.588463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591178+0000","last_change":"2024-09-21T02:09:11.536755+0000","last_active":"2024-09-21T02:09:16.591178+0000","last_peered":"2024-09-21T02:09:16.591178+0000","last_clean":"2024-09-21T02:09:16.591178+0000","last_became_active":"2024-09-21T02:09:11.535871+0000","last_became_peered":"2024-09-21T02:09:11.535871+0000","last_unstale":"2024-09-21T02:09:16.591178+0000","last_undegraded":"2024-09-21T02:09:16.591178+0000","last_fullsized":"2024-09-21T02:09:16.591178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:00.938047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724464+0000","last_change":"2024-09-21T02:09:11.572153+0000","last_active":"2024-09-21T02:09:16.724464+0000","last_peered":"2024-09-21T02:09:16.724464+0000","last_clean":"2024-09-21T02:09:16.724464+0000","last_became_active":"2024-09-21T02:09:11.567470+0000","last_became_peered":"2024-09-21T02:09:11.567470+0000","last_unstale":"2024-09-21T02:09:16.724464+0000","last_undegraded":"2024-09-21T02:09:16.724464+0000","last_fullsized":"2024-09-21T02:09:16.724464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:24.775708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591826+0000","last_change":"2024-09-21T02:09:11.532323+0000","last_active":"2024-09-21T02:09:16.591826+0000","last_peered":"2024-09-21T02:09:16.591826+0000","last_clean":"2024-09-21T02:09:16.591826+0000","last_became_active":"2024-09-21T02:09:11.531480+0000","last_became_peered":"2024-09-21T02:09:11.531480+0000","last_unstale":"2024-09-21T02:09:16.591826+0000","last_undegraded":"2024-09-21T02:09:16.591826+0000","last_fullsized":"2024-09-21T02:09:16.591826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:40.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723696+0000","last_change":"2024-09-21T02:09:11.575204+0000","last_active":"2024-09-21T02:09:16.723696+0000","last_peered":"2024-09-21T02:09:16.723696+0000","last_clean":"2024-09-21T02:09:16.723696+0000","last_became_active":"2024-09-21T02:09:11.567379+0000","last_became_peered":"2024-09-21T02:09:11.567379+0000","last_unstale":"2024-09-21T02:09:16.723696+0000","last_undegraded":"2024-09-21T02:09:16.723696+0000","last_fullsized":"2024-09-21T02:09:16.723696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:48.813808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722559+0000","last_change":"2024-09-21T02:09:11.540599+0000","last_active":"2024-09-21T02:09:16.722559+0000","last_peered":"2024-09-21T02:09:16.722559+0000","last_clean":"2024-09-21T02:09:16.722559+0000","last_became_active":"2024-09-21T02:09:11.539829+0000","last_became_peered":"2024-09-21T02:09:11.539829+0000","last_unstale":"2024-09-21T02:09:16.722559+0000","last_undegraded":"2024-09-21T02:09:16.722559+0000","last_fullsized":"2024-09-21T02:09:16.722559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:54.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719555+0000","last_change":"2024-09-21T02:09:11.534649+0000","last_active":"2024-09-21T02:09:16.719555+0000","last_peered":"2024-09-21T02:09:16.719555+0000","last_clean":"2024-09-21T02:09:16.719555+0000","last_became_active":"2024-09-21T02:09:11.533392+0000","last_became_peered":"2024-09-21T02:09:11.533392+0000","last_unstale":"2024-09-21T02:09:16.719555+0000","last_undegraded":"2024-09-21T02:09:16.719555+0000","last_fullsized":"2024-09-21T02:09:16.719555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592021+0000","last_change":"2024-09-21T02:09:11.546630+0000","last_active":"2024-09-21T02:09:16.592021+0000","last_peered":"2024-09-21T02:09:16.592021+0000","last_clean":"2024-09-21T02:09:16.592021+0000","last_became_active":"2024-09-21T02:09:11.545498+0000","last_became_peered":"2024-09-21T02:09:11.545498+0000","last_unstale":"2024-09-21T02:09:16.592021+0000","last_undegraded":"2024-09-21T02:09:16.592021+0000","last_fullsized":"2024-09-21T02:09:16.592021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:25.840553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":699032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":108172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":108082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":282624,"data_stored":108242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":699088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:09:34.133 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:09:34.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T02:09:34.134 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 -- 172.21.15.96:0/1705279523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd015f9a0 msgr2=0x7fbbd00d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/1705279523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd015f9a0 0x7fbbd00d2460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009900 tx=0x7fbbc401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 -- 172.21.15.96:0/1705279523 shutdown_connections 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/1705279523 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbbd00d29a0 0x7fbbd00d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/1705279523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd015f9a0 0x7fbbd00d2460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 -- 172.21.15.96:0/1705279523 >> 172.21.15.96:0/1705279523 conn(0x7fbbd00d1b10 msgr2=0x7fbbd00d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 -- 172.21.15.96:0/1705279523 shutdown_connections 2024-09-21T02:09:34.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.316+0000 7fbbd68a0640 1 -- 172.21.15.96:0/1705279523 wait complete. 2024-09-21T02:09:34.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 Processor -- start 2024-09-21T02:09:34.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 -- start start 2024-09-21T02:09:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 0x7fbbd01588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd00d29a0 0x7fbbd014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbbd015f9a0 0x7fbbd0157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd016c140 con 0x7fbbd00cf710 2024-09-21T02:09:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd016bfc0 con 0x7fbbd00d29a0 2024-09-21T02:09:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd68a0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd016c2c0 con 0x7fbbd015f9a0 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 0x7fbbd01588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 0x7fbbd01588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59720/0 (socket says 172.21.15.96:59720) 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 -- 172.21.15.96:0/3931039832 learned_addr learned my addr 172.21.15.96:0/3931039832 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcf7fe640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd00d29a0 0x7fbbd014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbd4e3d640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbbd015f9a0 0x7fbbd0157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 -- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbbd015f9a0 msgr2=0x7fbbd0157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbbd015f9a0 0x7fbbd0157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 -- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd00d29a0 msgr2=0x7fbbd014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd00d29a0 0x7fbbd014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 -- 172.21.15.96:0/3931039832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd014dcf0 con 0x7fbbd00cf710 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.320+0000 7fbbcffff640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 0x7fbbd01588f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c3f0 tx=0x7fbbc000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0017070 con 0x7fbbd00cf710 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbc00053c0 con 0x7fbbd00cf710 2024-09-21T02:09:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd00ce520 con 0x7fbbd00cf710 2024-09-21T02:09:34.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0003860 con 0x7fbbd00cf710 2024-09-21T02:09:34.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd01573a0 con 0x7fbbd00cf710 2024-09-21T02:09:34.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd00b59e0 con 0x7fbbd00cf710 2024-09-21T02:09:34.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbc001a070 con 0x7fbbd00cf710 2024-09-21T02:09:34.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbccff9640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbbbc073200 0x7fbbbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbbc0093840 con 0x7fbbd00cf710 2024-09-21T02:09:34.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbcf7fe640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbbbc073200 0x7fbbbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.324+0000 7fbbcf7fe640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbbbc073200 0x7fbbbc075710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40af000 tx=0x7fbbc401b120 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.328+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbbc0001090 con 0x7fbbd00cf710 2024-09-21T02:09:34.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.500+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fbbd0156e70 con 0x7fbbd00cf710 2024-09-21T02:09:34.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbccff9640 1 -- 172.21.15.96:0/3931039832 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fbbc005e5c0 con 0x7fbbd00cf710 2024-09-21T02:09:34.507 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbbbc073200 msgr2=0x7fbbbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbbbc073200 0x7fbbbc075710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40af000 tx=0x7fbbc401b120 comp rx=0 tx=0).stop 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 msgr2=0x7fbbd01588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 0x7fbbd01588f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c3f0 tx=0x7fbbc000c8b0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 shutdown_connections 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbbbc073200 0x7fbbbc075710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbbd015f9a0 0x7fbbd0157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd00d29a0 0x7fbbd014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 --2- 172.21.15.96:0/3931039832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbd00cf710 0x7fbbd01588f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 >> 172.21.15.96:0/3931039832 conn(0x7fbbd00d1b10 msgr2=0x7fbbd0068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 shutdown_connections 2024-09-21T02:09:34.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.504+0000 7fbbd68a0640 1 -- 172.21.15.96:0/3931039832 wait complete. 2024-09-21T02:09:34.528 INFO:teuthology.orchestra.run.smithi096.stdout:[{"version":31,"timestamp":"2024-09-21T02:08:58.964871+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"5936.331421"}]}] 2024-09-21T02:09:34.529 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T02:09:34.529 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T02:09:34.529 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T02:09:34.529 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T02:09:34.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- 172.21.15.96:0/603002608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f984006dea0 msgr2=0x7f984006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 --2- 172.21.15.96:0/603002608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f984006dea0 0x7f984006e2b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f98400cdb80 tx=0x7f983401c7d0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- 172.21.15.96:0/603002608 shutdown_connections 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 --2- 172.21.15.96:0/603002608 >> v2:172.21.15.122:3300/0 conn(0x7f98400cab60 0x7f98400caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 --2- 172.21.15.96:0/603002608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f984006dea0 0x7f984006e2b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- 172.21.15.96:0/603002608 >> 172.21.15.96:0/603002608 conn(0x7f984006eea0 msgr2=0x7f984006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- 172.21.15.96:0/603002608 shutdown_connections 2024-09-21T02:09:34.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- 172.21.15.96:0/603002608 wait complete. 2024-09-21T02:09:34.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 Processor -- start 2024-09-21T02:09:34.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- start start 2024-09-21T02:09:34.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f984006dea0 0x7f98401fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98400cab60 0x7f984014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 0x7f984006f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9840168390 con 0x7f9840155240 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9840168210 con 0x7f984006dea0 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f9846e7e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9840168510 con 0x7f98400cab60 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.668+0000 7f984541b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 0x7f984006f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 0x7f984006f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59738/0 (socket says 172.21.15.96:59738) 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 -- 172.21.15.96:0/218694210 learned_addr learned my addr 172.21.15.96:0/218694210 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f9844c1a640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f984006dea0 0x7f98401fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 -- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98400cab60 msgr2=0x7f984014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98400cab60 0x7f984014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 -- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f984006dea0 msgr2=0x7f98401fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f984006dea0 0x7f98401fa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 -- 172.21.15.96:0/218694210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984014fd20 con 0x7f9840155240 2024-09-21T02:09:34.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f984541b640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 0x7f984006f2b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f983800ca60 tx=0x7f983800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838007d90 con 0x7f9840155240 2024-09-21T02:09:34.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9838004c20 con 0x7f9840155240 2024-09-21T02:09:34.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9840150dd0 con 0x7f9840155240 2024-09-21T02:09:34.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838005280 con 0x7f9840155240 2024-09-21T02:09:34.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984014ffc0 con 0x7f9840155240 2024-09-21T02:09:34.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984006e2b0 con 0x7f9840155240 2024-09-21T02:09:34.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9838005420 con 0x7f9840155240 2024-09-21T02:09:34.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f983d7fa640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f982c073200 0x7f982c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9838094cc0 con 0x7f9840155240 2024-09-21T02:09:34.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.672+0000 7f9844c1a640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f982c073200 0x7f982c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.676+0000 7f9844c1a640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f982c073200 0x7f982c075710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9830000f70 tx=0x7f9830009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.676+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9838014030 con 0x7f9840155240 2024-09-21T02:09:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.888+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9840151000 con 0x7f9840155240 2024-09-21T02:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f983d7fa640 1 -- 172.21.15.96:0/218694210 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186459 (secure 0 0 0) 0x7f983805fa30 con 0x7f9840155240 2024-09-21T02:09:34.901 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:34.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f982c073200 msgr2=0x7f982c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f982c073200 0x7f982c075710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9830000f70 tx=0x7f9830009040 comp rx=0 tx=0).stop 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 msgr2=0x7f984006f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 0x7f984006f2b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f983800ca60 tx=0x7f983800cf20 comp rx=0 tx=0).stop 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 shutdown_connections 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f982c073200 0x7f982c075710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840155240 0x7f984006f2b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98400cab60 0x7f984014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 --2- 172.21.15.96:0/218694210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f984006dea0 0x7f98401fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.896+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 >> 172.21.15.96:0/218694210 conn(0x7f984006eea0 msgr2=0x7f984006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.900+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 shutdown_connections 2024-09-21T02:09:34.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:34.900+0000 7f9846e7e640 1 -- 172.21.15.96:0/218694210 wait complete. 2024-09-21T02:09:34.921 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"flags":0,"active_gid":4139,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3701893359},{"type":"v1","addr":"172.21.15.122:6801","nonce":3701893359}]},"active_addr":"172.21.15.122:6801/3701893359","active_change":"2024-09-21T02:08:50.479739+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4182,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":58541483}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2698644521}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":7671139}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":4117112254}]}]} 2024-09-21T02:09:34.925 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T02:09:34.925 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T02:09:34.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 -- 172.21.15.96:0/177657975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 msgr2=0x7f3c0c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 --2- 172.21.15.96:0/177657975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c06dea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009900 tx=0x7f3bfc01c8b0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 -- 172.21.15.96:0/177657975 shutdown_connections 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 --2- 172.21.15.96:0/177657975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c06dea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 --2- 172.21.15.96:0/177657975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c0c155240 0x7f3c0c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 -- 172.21.15.96:0/177657975 >> 172.21.15.96:0/177657975 conn(0x7f3c0c06eea0 msgr2=0x7f3c0c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 -- 172.21.15.96:0/177657975 shutdown_connections 2024-09-21T02:09:35.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 -- 172.21.15.96:0/177657975 wait complete. 2024-09-21T02:09:35.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.060+0000 7f3c14439640 1 Processor -- start 2024-09-21T02:09:35.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- start start 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c0c06e3e0 0x7f3c0c148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c1f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c0c155240 0x7f3c0c14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c16a220 con 0x7f3c0c073970 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c16a0a0 con 0x7f3c0c155240 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c16a3a0 con 0x7f3c0c06e3e0 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c1f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c1f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59766/0 (socket says 172.21.15.96:59766) 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 -- 172.21.15.96:0/1899103415 learned_addr learned my addr 172.21.15.96:0/1899103415 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 -- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c0c06e3e0 msgr2=0x7f3c0c148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c0c06e3e0 0x7f3c0c148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 -- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c0c155240 msgr2=0x7f3c0c14d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c0c155240 0x7f3c0c14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 -- 172.21.15.96:0/1899103415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c149930 con 0x7f3c0c073970 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c119d4640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c1f8450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009850 tx=0x7f3bfc0a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004040 con 0x7f3c0c073970 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bfc0042d0 con 0x7f3c0c073970 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c14e120 con 0x7f3c0c073970 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003860 con 0x7f3c0c073970 2024-09-21T02:09:35.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1f7c50 con 0x7f3c0c073970 2024-09-21T02:09:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c14de90 con 0x7f3c0c073970 2024-09-21T02:09:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.064+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bfc0c0030 con 0x7f3c0c073970 2024-09-21T02:09:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.068+0000 7f3c02ffd640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3bf0073230 0x7f3bf0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.068+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3bfc12e940 con 0x7f3c0c073970 2024-09-21T02:09:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.068+0000 7f3c121d5640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3bf0073230 0x7f3bf0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.068+0000 7f3c121d5640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3bf0073230 0x7f3bf0075740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c080058d0 tx=0x7f3c08005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.072+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3bfc0c7060 con 0x7f3c0c073970 2024-09-21T02:09:35.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.240+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c0c06dea0 con 0x7f3c0c073970 2024-09-21T02:09:35.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.240+0000 7f3c02ffd640 1 -- 172.21.15.96:0/1899103415 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7f3c0c149930 con 0x7f3c0c073970 2024-09-21T02:09:35.246 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:35.246 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:35.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3bf0073230 msgr2=0x7f3bf0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3bf0073230 0x7f3bf0075740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c080058d0 tx=0x7f3c08005860 comp rx=0 tx=0).stop 2024-09-21T02:09:35.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 msgr2=0x7f3c0c1f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c1f8450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009850 tx=0x7f3bfc0a6660 comp rx=0 tx=0).stop 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 shutdown_connections 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3bf0073230 0x7f3bf0075740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c0c155240 0x7f3c0c14d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c0c073970 0x7f3c0c1f8450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 --2- 172.21.15.96:0/1899103415 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c0c06e3e0 0x7f3c0c148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 >> 172.21.15.96:0/1899103415 conn(0x7f3c0c06eea0 msgr2=0x7f3c0c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 shutdown_connections 2024-09-21T02:09:35.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.244+0000 7f3c14439640 1 -- 172.21.15.96:0/1899103415 wait complete. 2024-09-21T02:09:35.266 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T02:09:35.266 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 -- 172.21.15.96:0/1613255039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 msgr2=0x7fc27015ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 --2- 172.21.15.96:0/1613255039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc27015ff90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc264009c80 tx=0x7fc26401cd10 comp rx=0 tx=0).stop 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 -- 172.21.15.96:0/1613255039 shutdown_connections 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 --2- 172.21.15.96:0/1613255039 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2701604d0 0x7fc270152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 --2- 172.21.15.96:0/1613255039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc27015ff90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 -- 172.21.15.96:0/1613255039 >> 172.21.15.96:0/1613255039 conn(0x7fc27006fbe0 msgr2=0x7fc27006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 -- 172.21.15.96:0/1613255039 shutdown_connections 2024-09-21T02:09:35.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.408+0000 7fc2751a8640 1 -- 172.21.15.96:0/1613255039 wait complete. 2024-09-21T02:09:35.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 Processor -- start 2024-09-21T02:09:35.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- start start 2024-09-21T02:09:35.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2700cce40 0x7fc2700c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc2701454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2701604d0 0x7fc27014afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700cc1f0 con 0x7fc2700cd250 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700cc070 con 0x7fc2701604d0 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc2700cc370 con 0x7fc2700cce40 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc2701454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc2701454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59778/0 (socket says 172.21.15.96:59778) 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 -- 172.21.15.96:0/4231192772 learned_addr learned my addr 172.21.15.96:0/4231192772 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26ed9d640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2700cce40 0x7fc2700c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26f59e640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2701604d0 0x7fc27014afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 -- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2700cce40 msgr2=0x7fc2700c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2700cce40 0x7fc2700c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 -- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2701604d0 msgr2=0x7fc27014afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2701604d0 0x7fc27014afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 -- 172.21.15.96:0/4231192772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700c5b40 con 0x7fc2700cd250 2024-09-21T02:09:35.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26ed9d640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2700cce40 0x7fc2700c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26f59e640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2701604d0 0x7fc27014afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc26e59c640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc2701454e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc264009bd0 tx=0x7fc264009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2640bb070 con 0x7fc2700cd250 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc264004d40 con 0x7fc2700cd250 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2640b2040 con 0x7fc2700cd250 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27014a260 con 0x7fc2700cd250 2024-09-21T02:09:35.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27014a7e0 con 0x7fc2700cd250 2024-09-21T02:09:35.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.412+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2700cb030 con 0x7fc2700cd250 2024-09-21T02:09:35.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.416+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc2640be070 con 0x7fc2700cd250 2024-09-21T02:09:35.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.416+0000 7fc2577fe640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc25c073230 0x7fc25c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.416+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc26412e2f0 con 0x7fc2700cd250 2024-09-21T02:09:35.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.416+0000 7fc26ed9d640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc25c073230 0x7fc25c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.416+0000 7fc26ed9d640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc25c073230 0x7fc25c075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc268001630 tx=0x7fc268009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.420+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc264016990 con 0x7fc2700cd250 2024-09-21T02:09:35.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.584+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc270150360 con 0x7fc2700cd250 2024-09-21T02:09:35.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2577fe640 1 -- 172.21.15.96:0/4231192772 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7fc2640f8fe0 con 0x7fc2700cd250 2024-09-21T02:09:35.591 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:35.591 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:35.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc25c073230 msgr2=0x7fc25c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc25c073230 0x7fc25c075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc268001630 tx=0x7fc268009040 comp rx=0 tx=0).stop 2024-09-21T02:09:35.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 msgr2=0x7fc2701454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc2701454e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc264009bd0 tx=0x7fc264009360 comp rx=0 tx=0).stop 2024-09-21T02:09:35.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 shutdown_connections 2024-09-21T02:09:35.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc25c073230 0x7fc25c075740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2701604d0 0x7fc27014afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2700cd250 0x7fc2701454e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 --2- 172.21.15.96:0/4231192772 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2700cce40 0x7fc2700c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 >> 172.21.15.96:0/4231192772 conn(0x7fc27006fbe0 msgr2=0x7fc2700b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 shutdown_connections 2024-09-21T02:09:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.588+0000 7fc2751a8640 1 -- 172.21.15.96:0/4231192772 wait complete. 2024-09-21T02:09:35.611 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T02:09:35.611 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T02:09:35.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T02:09:35.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T02:09:35.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T02:09:35.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T02:09:35.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T02:09:35.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T02:09:35.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 -- 172.21.15.96:0/3031117399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefcc0e2ba0 msgr2=0x7fefcc005d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 --2- 172.21.15.96:0/3031117399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefcc0e2ba0 0x7fefcc005d20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fefd4051240 tx=0x7fefd40983d0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 -- 172.21.15.96:0/3031117399 shutdown_connections 2024-09-21T02:09:35.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 --2- 172.21.15.96:0/3031117399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefcc0e2ba0 0x7fefcc005d20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 --2- 172.21.15.96:0/3031117399 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fefcc036ac0 0x7fefcc00afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 -- 172.21.15.96:0/3031117399 >> 172.21.15.96:0/3031117399 conn(0x7fefcc0513f0 msgr2=0x7fefcc051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 -- 172.21.15.96:0/3448013377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 msgr2=0x7fc5e015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 --2- 172.21.15.96:0/3448013377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 0x7fc5e015c250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009c80 tx=0x7fc5d801cd70 comp rx=0 tx=0).stop 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 -- 172.21.15.96:0/3448013377 shutdown_connections 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 --2- 172.21.15.96:0/3448013377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 0x7fc5e015c250 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 -- 172.21.15.96:0/3448013377 >> 172.21.15.96:0/3448013377 conn(0x7fc5e006ce90 msgr2=0x7fc5e006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 -- 172.21.15.96:0/3448013377 shutdown_connections 2024-09-21T02:09:35.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fc5e694d640 1 -- 172.21.15.96:0/3448013377 wait complete. 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 -- 172.21.15.96:0/3235305793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d4073080 msgr2=0x7f42d4073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 --2- 172.21.15.96:0/3235305793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d4073080 0x7f42d4073490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009980 tx=0x7f42c401c6b0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 -- 172.21.15.96:0/3235305793 shutdown_connections 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 --2- 172.21.15.96:0/3235305793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d4073080 0x7f42d4073490 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 --2- 172.21.15.96:0/3235305793 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42d408b290 0x7f42d408b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 -- 172.21.15.96:0/3235305793 >> 172.21.15.96:0/3235305793 conn(0x7f42d40d0ad0 msgr2=0x7f42d406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 -- 172.21.15.96:0/3235305793 shutdown_connections 2024-09-21T02:09:35.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.832+0000 7fefd9281640 1 -- 172.21.15.96:0/3031117399 shutdown_connections 2024-09-21T02:09:35.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7fc5e694d640 1 Processor -- start 2024-09-21T02:09:35.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7fc5e694d640 1 -- start start 2024-09-21T02:09:35.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 -- 172.21.15.96:0/3235305793 wait complete. 2024-09-21T02:09:35.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 Processor -- start 2024-09-21T02:09:35.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7fefd9281640 1 -- 172.21.15.96:0/3031117399 wait complete. 2024-09-21T02:09:35.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7fbf56231640 1 -- 172.21.15.96:0/3171533067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c004810 msgr2=0x7fbf3c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7fbf56231640 1 --2- 172.21.15.96:0/3171533067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c004810 0x7fbf3c004c20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ab00 tx=0x7fbf4c01cb90 comp rx=0 tx=0).stop 2024-09-21T02:09:35.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7fbf55a30640 1 -- 172.21.15.96:0/3171533067 reap_dead start 2024-09-21T02:09:35.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5e00700e0 0x7fc5e0208390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 0x7fc5e014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e014f970 0x7fc5e0153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5e00700e0 0x7fc5e0208390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5e00700e0 0x7fc5e0208390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34366/0 (socket says 172.21.15.96:34366) 2024-09-21T02:09:35.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 Processor -- start 2024-09-21T02:09:35.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e016d590 con 0x7fc5e00cef70 2024-09-21T02:09:35.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 -- 172.21.15.96:0/3374092846 learned_addr learned my addr 172.21.15.96:0/3374092846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e016d410 con 0x7fc5e014f970 2024-09-21T02:09:35.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e016d710 con 0x7fc5e00700e0 2024-09-21T02:09:35.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 -- start start 2024-09-21T02:09:35.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e614c640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e014f970 0x7fc5e0153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.836+0000 7f42da6b4640 1 -- start start 2024-09-21T02:09:35.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42d4073080 0x7f42d40a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42d408b290 0x7f42d408ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e514a640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 0x7fc5e014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d416b1a0 0x7f42d416d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4178670 con 0x7f42d416b1a0 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41784f0 con 0x7f42d4073080 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d37fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42d408b290 0x7f42d408ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f42d41787f0 con 0x7f42d408b290 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d37fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42d408b290 0x7f42d408ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34368/0 (socket says 172.21.15.96:34368) 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d37fe640 1 -- 172.21.15.96:0/2355362627 learned_addr learned my addr 172.21.15.96:0/2355362627 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d416b1a0 0x7f42d416d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 -- 172.21.15.96:0/3573143449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 msgr2=0x7fc79815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/3573143449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 0x7fc79815e7f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc798071a50 tx=0x7fc79401d500 comp rx=0 tx=0).stop 2024-09-21T02:09:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d3fff640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42d4073080 0x7f42d40a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 -- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e014f970 msgr2=0x7fc5e0153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e014f970 0x7fc5e0153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 -- 172.21.15.96:0/3374092846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 msgr2=0x7fc5e014f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5e00cef70 0x7fc5e014f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e014fd80 con 0x7fc5e00700e0 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 -- 172.21.15.96:0/3573143449 shutdown_connections 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/3573143449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 0x7fc79815e7f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/3573143449 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc79806dea0 0x7fc79806e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/3573143449 >> v2:172.21.15.122:3300/0 conn(0x7fc798155040 0x7fc7980ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 -- 172.21.15.96:0/3573143449 >> 172.21.15.96:0/3573143449 conn(0x7fc79806eea0 msgr2=0x7fc79806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 -- 172.21.15.96:0/3573143449 shutdown_connections 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 -- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42d408b290 msgr2=0x7f42d408ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42d408b290 0x7f42d408ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 -- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42d4073080 msgr2=0x7f42d40a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42d4073080 0x7f42d40a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 -- 172.21.15.96:0/2355362627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d41659f0 con 0x7f42d416b1a0 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e594b640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5e00700e0 0x7fc5e0208390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801a9a0 tx=0x7fc5d80a5da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 -- 172.21.15.96:0/4157386461 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140cac90 msgr2=0x7f2d1406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 --2- 172.21.15.96:0/4157386461 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140cac90 0x7f2d1406e3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009c80 tx=0x7f2d0c01cda0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 -- 172.21.15.96:0/4157386461 shutdown_connections 2024-09-21T02:09:35.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 --2- 172.21.15.96:0/4157386461 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140cac90 0x7f2d1406e3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 --2- 172.21.15.96:0/4157386461 >> v2:172.21.15.122:3300/0 conn(0x7f2d140ca880 0x7f2d1406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 -- 172.21.15.96:0/4157386461 >> 172.21.15.96:0/4157386461 conn(0x7f2d1406eea0 msgr2=0x7f2d1406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fbf56231640 1 -- 172.21.15.96:0/3171533067 shutdown_connections 2024-09-21T02:09:35.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fbf56231640 1 -- 172.21.15.96:0/3171533067 >> 172.21.15.96:0/3171533067 conn(0x7fbf500b1a30 msgr2=0x7fbf500b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d8c51640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d416b1a0 0x7f42d416d260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f42d4073010 tx=0x7f42c40aa880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b2040 con 0x7fc5e00700e0 2024-09-21T02:09:35.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 -- 172.21.15.96:0/4157386461 shutdown_connections 2024-09-21T02:09:35.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d3fff640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42d4073080 0x7f42d40a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:35.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e0152410 con 0x7fc5e00700e0 2024-09-21T02:09:35.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e00b5b00 con 0x7fc5e00700e0 2024-09-21T02:09:35.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fbf56231640 1 -- 172.21.15.96:0/3171533067 shutdown_connections 2024-09-21T02:09:35.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fbf56231640 1 -- 172.21.15.96:0/3171533067 wait complete. 2024-09-21T02:09:35.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5d8003b80 con 0x7fc5e00700e0 2024-09-21T02:09:35.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80ae780 con 0x7fc5e00700e0 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c40aad30 con 0x7f42d416b1a0 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42c40bc070 con 0x7f42d416b1a0 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4003860 con 0x7f42d416b1a0 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 -- 172.21.15.96:0/2355362627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d4165cd0 con 0x7f42d416b1a0 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefcc0df240 0x7fefcc0ec940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefcc0e2ba0 0x7fefcc0532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fefcc0de780 0x7fefcc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc0e3ec0 con 0x7fefcc0e2ba0 2024-09-21T02:09:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc0e3d40 con 0x7fefcc0df240 2024-09-21T02:09:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fefd9281640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc0e4040 con 0x7fefcc0de780 2024-09-21T02:09:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f2d1a60d640 1 -- 172.21.15.96:0/4157386461 wait complete. 2024-09-21T02:09:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7f42da6b4640 1 -- 172.21.15.96:0/2355362627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d4166290 con 0x7f42d416b1a0 2024-09-21T02:09:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5e015c250 con 0x7fc5e00700e0 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 -- 172.21.15.96:0/3573143449 wait complete. 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.840+0000 7fc79fa1b640 1 Processor -- start 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 -- start start 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefcc0df240 0x7fefcc0ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefcc0df240 0x7fefcc0ec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.96:56548/0 (socket says 172.21.15.96:56548) 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 -- 172.21.15.96:0/3469367478 learned_addr learned my addr 172.21.15.96:0/3469367478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42ac001040 con 0x7f42d416b1a0 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fbf56231640 1 Processor -- start 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 -- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fefcc0de780 msgr2=0x7fefcc0537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fefcc0de780 0x7fefcc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 -- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefcc0e2ba0 msgr2=0x7fefcc0532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefcc0e2ba0 0x7fefcc0532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 -- 172.21.15.96:0/3469367478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc0eae10 con 0x7fefcc0df240 2024-09-21T02:09:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f2d1a60d640 1 Processor -- start 2024-09-21T02:09:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd2ffd640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefcc0df240 0x7fefcc0ec940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fefc400c490 tx=0x7fefc400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc79806dea0 0x7fc7981f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f2d1a60d640 1 -- start start 2024-09-21T02:09:35.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc798155040 0x7fc7980c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79d7b7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc79806dea0 0x7fc7981f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79d7b7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc79806dea0 0x7fc7981f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34384/0 (socket says 172.21.15.96:34384) 2024-09-21T02:09:35.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79d7b7640 1 -- 172.21.15.96:0/1544166873 learned_addr learned my addr 172.21.15.96:0/1544166873 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79cfb6640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc798155040 0x7fc7980c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 0x7fc7981fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fbf56231640 1 -- start start 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980743a0 con 0x7fc79815e3e0 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5d80ae920 con 0x7fc5e00700e0 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5d67fc640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc5c4073230 0x7fc5c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc5d812f060 con 0x7fc5e00700e0 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5d67fc640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7fc5c407c9f0 0x7fc5c407ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5c407f550 con 0x7fc5c407c9f0 2024-09-21T02:09:35.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5d812f450 con 0x7fc5e00700e0 2024-09-21T02:09:35.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc798074220 con 0x7fc798155040 2024-09-21T02:09:35.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79dfb8640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 0x7fc7981fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc798074520 con 0x7fc79806dea0 2024-09-21T02:09:35.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbf3c004810 0x7fbf501461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf500ab630 0x7fbf500aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf501f88d0 0x7fbf501f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5006e3f0 con 0x7fbf500ab630 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5006e270 con 0x7fbf501f88d0 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5006e570 con 0x7fbf3c004810 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf500ab630 0x7fbf500aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf500ab630 0x7fbf500aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59902/0 (socket says 172.21.15.96:59902) 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 -- 172.21.15.96:0/1355230463 learned_addr learned my addr 172.21.15.96:0/1355230463 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf5522f640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbf3c004810 0x7fbf501461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf55a30640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf501f88d0 0x7fbf501f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 -- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbf3c004810 msgr2=0x7fbf501461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbf3c004810 0x7fbf501461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 -- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf501f88d0 msgr2=0x7fbf501f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf501f88d0 0x7fbf501f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 -- 172.21.15.96:0/1355230463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5015a670 con 0x7fbf500ab630 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf5522f640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbf3c004810 0x7fbf501461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf55a30640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf501f88d0 0x7fbf501f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf54a2e640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf500ab630 0x7fbf500aed00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b0040 tx=0x7fbf4c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4017070 con 0x7fefcc0df240 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc17d760 con 0x7fefcc0df240 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc17dd20 con 0x7fefcc0df240 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefc4003600 con 0x7fefcc0df240 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4003a30 con 0x7fefcc0df240 2024-09-21T02:09:35.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefb4001040 con 0x7fefcc0df240 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f42c40b0070 con 0x7f42d416b1a0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42d0ff9640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f42c0073300 0x7f42c0075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f42c4132f60 con 0x7f42d416b1a0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42d0ff9640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f42c007cad0 0x7f42c007ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 --> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42c007f630 con 0x7f42c007cad0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42c4133350 con 0x7f42d416b1a0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f42d37fe640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f42c007cad0 0x7f42c007ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f42d3fff640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f42c0073300 0x7f42c0075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f42d3fff640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f42c0073300 0x7f42c0075810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006fd0 tx=0x7f42c8008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f42d37fe640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f42c007cad0 0x7f42c007ef60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.844+0000 7fc5e614c640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7fc5c407c9f0 0x7fc5c407ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc5e514a640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc5c4073230 0x7fc5c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140ca880 0x7f2d14156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d140cac90 0x7f2d14156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d1415e3e0 0x7f2d140c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d140cac90 0x7f2d14156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d140cac90 0x7f2d14156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59918/0 (socket says 172.21.15.96:59918) 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14166290 con 0x7f2d140cac90 2024-09-21T02:09:35.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14166110 con 0x7f2d1415e3e0 2024-09-21T02:09:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 -- 172.21.15.96:0/2713321512 learned_addr learned my addr 172.21.15.96:0/2713321512 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2d14166410 con 0x7f2d140ca880 2024-09-21T02:09:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d13fff640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140ca880 0x7f2d14156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f32ba110640 1 -- 172.21.15.96:0/1194078418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 msgr2=0x7f329c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f32ba110640 1 --2- 172.21.15.96:0/1194078418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f329c00ba10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c330 tx=0x7f32a801d020 comp rx=0 tx=0).stop 2024-09-21T02:09:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0b2040 con 0x7fbf500ab630 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 -- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140ca880 msgr2=0x7f2d14156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d140ca880 0x7f2d14156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 -- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d1415e3e0 msgr2=0x7f2d140c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d1415e3e0 0x7f2d140c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d140b5810 con 0x7f2d140cac90 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc5e614c640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7fc5c407c9f0 0x7fc5c407ee80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf4c0bb070 con 0x7fbf500ab630 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c004c20 con 0x7fbf500ab630 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5014a920 con 0x7fbf500ab630 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5014a2d0 con 0x7fbf500ab630 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79cfb6640 1 -- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc79806dea0 msgr2=0x7fc7981f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79cfb6640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc79806dea0 0x7fc7981f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79cfb6640 1 -- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 msgr2=0x7fc7981fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79cfb6640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79815e3e0 0x7fc7981fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79cfb6640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7980c4c70 con 0x7fc798155040 2024-09-21T02:09:35.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc5e514a640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc5c4073230 0x7fc5c4075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5e01516e0 tx=0x7fc5d0009060 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d137fe640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d140cac90 0x7f2d14156d70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000f90 tx=0x7f2d0c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0b2040 con 0x7f2d140cac90 2024-09-21T02:09:35.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d0c004d40 con 0x7f2d140cac90 2024-09-21T02:09:35.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0ad7f0 con 0x7f2d140cac90 2024-09-21T02:09:35.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79cfb6640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc798155040 0x7fc7980c2940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc79000ca30 tx=0x7fc79000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f32ba110640 1 -- 172.21.15.96:0/1194078418 shutdown_connections 2024-09-21T02:09:35.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f32ba110640 1 --2- 172.21.15.96:0/1194078418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f329c00ba10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f32ba110640 1 -- 172.21.15.96:0/1194078418 >> 172.21.15.96:0/1194078418 conn(0x7f32b40d03f0 msgr2=0x7f32b40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790004d30 con 0x7fc798155040 2024-09-21T02:09:35.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc790007b30 con 0x7fc798155040 2024-09-21T02:09:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d140c9870 con 0x7f2d140cac90 2024-09-21T02:09:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d140b3330 con 0x7f2d140cac90 2024-09-21T02:09:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7980c4f50 con 0x7fc798155040 2024-09-21T02:09:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79001d060 con 0x7fc798155040 2024-09-21T02:09:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7980c5510 con 0x7fc798155040 2024-09-21T02:09:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.848+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf500cf240 con 0x7fbf500ab630 2024-09-21T02:09:35.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d141572b0 con 0x7f2d140cac90 2024-09-21T02:09:35.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f32ba110640 1 -- 172.21.15.96:0/1194078418 shutdown_connections 2024-09-21T02:09:35.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf4c0c0460 con 0x7fbf500ab630 2024-09-21T02:09:35.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefc401a070 con 0x7fefcc0df240 2024-09-21T02:09:35.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f32ba110640 1 -- 172.21.15.96:0/1194078418 wait complete. 2024-09-21T02:09:35.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== osd.5 v2:172.21.15.122:6810/4177593286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c007f630 con 0x7f42c007cad0 2024-09-21T02:09:35.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== osd.6 v2:172.21.15.122:6821/3298887226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5c407f550 con 0x7fc5c407c9f0 2024-09-21T02:09:35.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7980c4a20 con 0x7fc798155040 2024-09-21T02:09:35.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2d0c0bd060 con 0x7f2d140cac90 2024-09-21T02:09:35.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc790005020 con 0x7fc798155040 2024-09-21T02:09:35.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc78dffb640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc77c073230 0x7fc77c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d10ff9640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2d00073230 0x7f2d00075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc7900952a0 con 0x7fc798155040 2024-09-21T02:09:35.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc78dffb640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fc77c07c970 0x7fc77c07ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc77c07f4d0 con 0x7fc77c07c970 2024-09-21T02:09:35.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc790095690 con 0x7fc798155040 2024-09-21T02:09:35.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2d0c12e570 con 0x7f2d140cac90 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d13fff640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2d00073230 0x7f2d00075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d10ff9640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f2d0007c9f0 0x7f2d0007ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d0c0b3e50 con 0x7f2d0007c9f0 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf45ffb640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbf34073230 0x7fbf34075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbf4c12f350 con 0x7fbf500ab630 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf5522f640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbf34073230 0x7fbf34075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf45ffb640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7fbf3407c900 0x7fbf3407ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 --> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf4c0b5e50 con 0x7fbf3407c900 2024-09-21T02:09:35.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf4c136050 con 0x7fbf500ab630 2024-09-21T02:09:35.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf5522f640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbf34073230 0x7fbf34075740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000fc0 tx=0x7fbf40001550 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc79dfb8640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fc77c07c970 0x7fc77c07ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c135050 con 0x7f2d140cac90 2024-09-21T02:09:35.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7f2d18baa640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f2d0007c9f0 0x7f2d0007ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fc79dfb8640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fc77c07c970 0x7fc77c07ee00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f2d13fff640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2d00073230 0x7f2d00075740 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d080058d0 tx=0x7f2d08005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f2d18baa640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f2d0007c9f0 0x7f2d0007ee80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.852+0000 7fbf55a30640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7fbf3407c900 0x7fbf3407ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7fbf55a30640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7fbf3407c900 0x7fbf3407ed90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7fc79d7b7640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc77c073230 0x7fc77c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 Processor -- start 2024-09-21T02:09:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== osd.0 v2:172.21.15.96:6800/190673067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf4c0b5e50 con 0x7fbf3407c900 2024-09-21T02:09:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== osd.7 v2:172.21.15.122:6818/1732471888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d0c0b3e50 con 0x7f2d0007c9f0 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 -- start start 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f32b414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b41577e0 0x7f32b4162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32b41571b0 0x7f32b4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32b4166850 con 0x7f329c005360 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32b4073be0 con 0x7f32b41577e0 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f32b41669d0 con 0x7f32b41571b0 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f32b414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f32b414e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59920/0 (socket says 172.21.15.96:59920) 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 -- 172.21.15.96:0/2058529863 learned_addr learned my addr 172.21.15.96:0/2058529863 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b2ffd640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b41577e0 0x7f32b4162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== osd.1 v2:172.21.15.96:6816/2322744690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc77c07f4d0 con 0x7fc77c07c970 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 -- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32b41571b0 msgr2=0x7f32b4159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32b41571b0 0x7f32b4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 -- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b41577e0 msgr2=0x7f32b4162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b41577e0 0x7f32b4162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 -- 172.21.15.96:0/2058529863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4050f70 con 0x7f329c005360 2024-09-21T02:09:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7fc79d7b7640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc77c073230 0x7fc77c075740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc78800ab40 tx=0x7fc78800a2d0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32b37fe640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f32b414e2e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32a400b3c0 tx=0x7f32a400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4015020 con 0x7f329c005360 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4011920 con 0x7f329c005360 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4011d70 con 0x7f329c005360 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.856+0000 7f32ba110640 1 -- 172.21.15.96:0/2058529863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b41611e0 con 0x7f329c005360 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f32ba110640 1 -- 172.21.15.96:0/2058529863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b41617a0 con 0x7f329c005360 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f32ba110640 1 -- 172.21.15.96:0/2058529863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32b4162e10 con 0x7f329c005360 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7fe551e65640 1 -- 172.21.15.96:0/3948088708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ce90 msgr2=0x7fe54c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7fe551e65640 1 --2- 172.21.15.96:0/3948088708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ce90 0x7fe54c06d2a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009840 tx=0x7fe53c01c7b0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f32a4018020 con 0x7f329c005360 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f329bfff640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f32a00732d0 0x7f32a00757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f32a4094410 con 0x7f329c005360 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f32b2ffd640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f32a00732d0 0x7f32a00757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f329bfff640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7f32a007caa0 0x7f32a007ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 --> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32a007f600 con 0x7f32a007caa0 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32a409b050 con 0x7f329c005360 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.860+0000 7f32b3fff640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7f32a007caa0 0x7f32a007ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefbb7fe640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fefc00732d0 0x7fefc00757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fefc4093db0 con 0x7fefcc0df240 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefbb7fe640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fefc007caa0 0x7fefc007ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefd27fc640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fefc00732d0 0x7fefc00757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefd37fe640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fefc007caa0 0x7fefc007ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7f32b2ffd640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f32a00732d0 0x7f32a00757e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32b4157cc0 tx=0x7f32ac003d50 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7f32b3fff640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7f32a007caa0 0x7f32a007ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== osd.3 v2:172.21.15.96:6808/3409109307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32a007f600 con 0x7f32a007caa0 2024-09-21T02:09:35.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefd37fe640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fefc007caa0 0x7fefc007ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 --> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefc007f600 con 0x7fefc007caa0 2024-09-21T02:09:35.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefd27fc640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fefc00732d0 0x7fefc00757e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefcc005420 tx=0x7fefc8006c00 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fefc40941a0 con 0x7fefcc0df240 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.864+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== osd.2 v2:172.21.15.96:6824/1085819941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefc007f600 con 0x7fefc007caa0 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.868+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 --> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42ac002de0 con 0x7f42c007cad0 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.868+0000 7f42d0ff9640 1 -- 172.21.15.96:0/2355362627 <== osd.5 v2:172.21.15.122:6810/4177593286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f42ac002de0 con 0x7f42c007cad0 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f42c007cad0 msgr2=0x7f42c007ef60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f42c007cad0 0x7f42c007ef60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f42c0073300 msgr2=0x7f42c0075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f42c0073300 0x7f42c0075810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006fd0 tx=0x7f42c8008040 comp rx=0 tx=0).stop 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d416b1a0 msgr2=0x7f42d416d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 --2- 172.21.15.96:0/2355362627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42d416b1a0 0x7f42d416d260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f42d4073010 tx=0x7f42c40aa880 comp rx=0 tx=0).stop 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42d8c51640 1 -- 172.21.15.96:0/2355362627 reap_dead start 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 shutdown_connections 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 >> 172.21.15.96:0/2355362627 conn(0x7f42d40d0ad0 msgr2=0x7f42d40af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7fe551e65640 1 -- 172.21.15.96:0/3948088708 shutdown_connections 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7fe551e65640 1 --2- 172.21.15.96:0/3948088708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ce90 0x7fe54c06d2a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7fe551e65640 1 -- 172.21.15.96:0/3948088708 >> 172.21.15.96:0/3948088708 conn(0x7fe54c0cdbb0 msgr2=0x7fe54c0cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 shutdown_connections 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7f42ba7fc640 1 -- 172.21.15.96:0/2355362627 wait complete. 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7fe551e65640 1 -- 172.21.15.96:0/3948088708 shutdown_connections 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.872+0000 7fe551e65640 1 -- 172.21.15.96:0/3948088708 wait complete. 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe551e65640 1 Processor -- start 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe551e65640 1 -- start start 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe551e65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54c06ce90 0x7fe54c15d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54c06ce90 0x7fe54c15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54c06ce90 0x7fe54c15d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59934/0 (socket says 172.21.15.96:59934) 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe54b7fe640 1 -- 172.21.15.96:0/2042802230 learned_addr learned my addr 172.21.15.96:0/2042802230 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:35.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe551e65640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ffb0 0x7fe54c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe551e65640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe54c070f80 0x7fe54c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.876+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c162eb0 con 0x7fe54c06ce90 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c162d30 con 0x7fe54c06ffb0 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54affd640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ffb0 0x7fe54c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54bfff640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe54c070f80 0x7fe54c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c163030 con 0x7fe54c070f80 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54b7fe640 1 -- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe54c070f80 msgr2=0x7fe54c15eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54b7fe640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe54c070f80 0x7fe54c15eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54b7fe640 1 -- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ffb0 msgr2=0x7fe54c153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54b7fe640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ffb0 0x7fe54c153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54b7fe640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c153cb0 con 0x7fe54c06ce90 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54affd640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe54c06ffb0 0x7fe54c153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe54b7fe640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54c06ce90 0x7fe54c15d870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe54400cbf0 tx=0x7fe54400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544016020 con 0x7fe54c06ce90 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe54400ab00 con 0x7fe54c06ce90 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544012420 con 0x7fe54c06ce90 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c15c810 con 0x7fe54c06ce90 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 --> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf500b51e0 con 0x7fbf3407c900 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fbf45ffb640 1 -- 172.21.15.96:0/1355230463 <== osd.0 v2:172.21.15.96:6800/190673067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf500b51e0 con 0x7fbf3407c900 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.880+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c15cd90 con 0x7fe54c06ce90 2024-09-21T02:09:35.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe54c15d450 con 0x7fe54c06ce90 2024-09-21T02:09:35.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7fbf3407c900 msgr2=0x7fbf3407ed90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7fbf3407c900 0x7fbf3407ed90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 --> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5e0153cc0 con 0x7fc5c407c9f0 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbf34073230 msgr2=0x7fbf34075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbf34073230 0x7fbf34075740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000fc0 tx=0x7fbf40001550 comp rx=0 tx=0).stop 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf500ab630 msgr2=0x7fbf500aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 --2- 172.21.15.96:0/1355230463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf500ab630 0x7fbf500aed00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b0040 tx=0x7fbf4c0056a0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf55a30640 1 -- 172.21.15.96:0/1355230463 reap_dead start 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 shutdown_connections 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 >> 172.21.15.96:0/1355230463 conn(0x7fbf500b1a30 msgr2=0x7fbf500b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 shutdown_connections 2024-09-21T02:09:35.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fbf56231640 1 -- 172.21.15.96:0/1355230463 wait complete. 2024-09-21T02:09:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5d67fc640 1 -- 172.21.15.96:0/3374092846 <== osd.6 v2:172.21.15.122:6821/3298887226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5dc001390 con 0x7fc5c407c9f0 2024-09-21T02:09:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe544019020 con 0x7fe54c06ce90 2024-09-21T02:09:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe533fff640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe538073230 0x7fe538075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe54affd640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe538073230 0x7fe538075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe544099060 con 0x7fe54c06ce90 2024-09-21T02:09:35.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7fc5c407c9f0 msgr2=0x7fc5c407ee80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7fc5c407c9f0 0x7fc5c407ee80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe533fff640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7fe53807c970 0x7fe53807ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc5c4073230 msgr2=0x7fc5c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc5c4073230 0x7fc5c4075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5e01516e0 tx=0x7fc5d0009060 comp rx=0 tx=0).stop 2024-09-21T02:09:35.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe544012e50 con 0x7fe53807c970 2024-09-21T02:09:35.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe54405a680 con 0x7fe54c06ce90 2024-09-21T02:09:35.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5e00700e0 msgr2=0x7fc5e0208390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 --2- 172.21.15.96:0/3374092846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5e00700e0 0x7fc5e0208390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801a9a0 tx=0x7fc5d80a5da0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e614c640 1 -- 172.21.15.96:0/3374092846 reap_dead start 2024-09-21T02:09:35.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 shutdown_connections 2024-09-21T02:09:35.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 >> 172.21.15.96:0/3374092846 conn(0x7fc5e006ce90 msgr2=0x7fc5e00b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe54bfff640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7fe53807c970 0x7fe53807ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 shutdown_connections 2024-09-21T02:09:35.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fc5e694d640 1 -- 172.21.15.96:0/3374092846 wait complete. 2024-09-21T02:09:35.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe54affd640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe538073230 0x7fe538075740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0c7c90 tx=0x7fe540009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.884+0000 7fe54bfff640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7fe53807c970 0x7fe53807ee00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.888+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 --> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7981f8fa0 con 0x7fc77c07c970 2024-09-21T02:09:35.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.888+0000 7fc78dffb640 1 -- 172.21.15.96:0/1544166873 <== osd.1 v2:172.21.15.96:6816/2322744690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc7981f8fa0 con 0x7fc77c07c970 2024-09-21T02:09:35.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.888+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== osd.4 v2:172.21.15.122:6802/567414413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe544012e50 con 0x7fe53807c970 2024-09-21T02:09:35.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.888+0000 7f32ba110640 1 -- 172.21.15.96:0/2058529863 --> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32b4162880 con 0x7f32a007caa0 2024-09-21T02:09:35.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fc77c07c970 msgr2=0x7fc77c07ee00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fc77c07c970 0x7fc77c07ee00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7f329bfff640 1 -- 172.21.15.96:0/2058529863 <== osd.3 v2:172.21.15.96:6808/3409109307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f32b4162880 con 0x7f32a007caa0 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc77c073230 msgr2=0x7fc77c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc77c073230 0x7fc77c075740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc78800ab40 tx=0x7fc78800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc798155040 msgr2=0x7fc7980c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 --2- 172.21.15.96:0/1544166873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc798155040 0x7fc7980c2940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc79000ca30 tx=0x7fc79000cef0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79dfb8640 1 -- 172.21.15.96:0/1544166873 reap_dead start 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 shutdown_connections 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 >> 172.21.15.96:0/1544166873 conn(0x7fc79806eea0 msgr2=0x7fc7980b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 shutdown_connections 2024-09-21T02:09:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.892+0000 7fc79fa1b640 1 -- 172.21.15.96:0/1544166873 wait complete. 2024-09-21T02:09:35.904 INFO:teuthology.orchestra.run.smithi096.stdout:60129542155 2024-09-21T02:09:35.904 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T02:09:35.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 --> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fefb4002d20 con 0x7fefc007caa0 2024-09-21T02:09:35.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefbb7fe640 1 -- 172.21.15.96:0/3469367478 <== osd.2 v2:172.21.15.96:6824/1085819941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fefb4002d20 con 0x7fefc007caa0 2024-09-21T02:09:35.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fefc007caa0 msgr2=0x7fefc007ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fefc007caa0 0x7fefc007ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fefc00732d0 msgr2=0x7fefc00757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fefc00732d0 0x7fefc00757e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefcc005420 tx=0x7fefc8006c00 comp rx=0 tx=0).stop 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefcc0df240 msgr2=0x7fefcc0ec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd9281640 1 --2- 172.21.15.96:0/3469367478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefcc0df240 0x7fefcc0ec940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fefc400c490 tx=0x7fefc400c950 comp rx=0 tx=0).stop 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.896+0000 7fefd37fe640 1 -- 172.21.15.96:0/3469367478 reap_dead start 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7f32a007caa0 msgr2=0x7f32a007ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7f32a007caa0 0x7f32a007ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f32a00732d0 msgr2=0x7f32a00757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f32a00732d0 0x7f32a00757e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32b4157cc0 tx=0x7f32ac003d50 comp rx=0 tx=0).stop 2024-09-21T02:09:35.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 --> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d140c9620 con 0x7f2d0007c9f0 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 msgr2=0x7f32b414e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 --2- 172.21.15.96:0/2058529863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f329c005360 0x7f32b414e2e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32a400b3c0 tx=0x7f32a400b880 comp rx=0 tx=0).stop 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f32b3fff640 1 -- 172.21.15.96:0/2058529863 reap_dead start 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 shutdown_connections 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 >> 172.21.15.96:0/2058529863 conn(0x7f32b40d03f0 msgr2=0x7f32b40cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 shutdown_connections 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f3299ffb640 1 -- 172.21.15.96:0/2058529863 wait complete. 2024-09-21T02:09:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d10ff9640 1 -- 172.21.15.96:0/2713321512 <== osd.7 v2:172.21.15.122:6818/1732471888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d140c9620 con 0x7f2d0007c9f0 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f2d0007c9f0 msgr2=0x7f2d0007ee80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f2d0007c9f0 0x7f2d0007ee80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2d00073230 msgr2=0x7f2d00075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2d00073230 0x7f2d00075740 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d080058d0 tx=0x7f2d08005860 comp rx=0 tx=0).stop 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d140cac90 msgr2=0x7f2d14156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 --2- 172.21.15.96:0/2713321512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d140cac90 0x7f2d14156d70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000f90 tx=0x7f2d0c009360 comp rx=0 tx=0).stop 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d18baa640 1 -- 172.21.15.96:0/2713321512 reap_dead start 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 shutdown_connections 2024-09-21T02:09:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 >> 172.21.15.96:0/2713321512 conn(0x7f2d1406eea0 msgr2=0x7f2d140ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 shutdown_connections 2024-09-21T02:09:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.900+0000 7f2d1a60d640 1 -- 172.21.15.96:0/2713321512 wait complete. 2024-09-21T02:09:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.904+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 shutdown_connections 2024-09-21T02:09:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.904+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 >> 172.21.15.96:0/3469367478 conn(0x7fefcc0513f0 msgr2=0x7fefcc052890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.904+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 shutdown_connections 2024-09-21T02:09:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.904+0000 7fefd9281640 1 -- 172.21.15.96:0/3469367478 wait complete. 2024-09-21T02:09:35.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 --> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe54c06d2a0 con 0x7fe53807c970 2024-09-21T02:09:35.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe533fff640 1 -- 172.21.15.96:0/2042802230 <== osd.4 v2:172.21.15.122:6802/567414413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe54c06d2a0 con 0x7fe53807c970 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7fe53807c970 msgr2=0x7fe53807ee00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7fe53807c970 0x7fe53807ee00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe538073230 msgr2=0x7fe538075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe538073230 0x7fe538075740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0c7c90 tx=0x7fe540009040 comp rx=0 tx=0).stop 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54c06ce90 msgr2=0x7fe54c15d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 --2- 172.21.15.96:0/2042802230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54c06ce90 0x7fe54c15d870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe54400cbf0 tx=0x7fe54400cce0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe54bfff640 1 -- 172.21.15.96:0/2042802230 reap_dead start 2024-09-21T02:09:35.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 shutdown_connections 2024-09-21T02:09:35.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 >> 172.21.15.96:0/2042802230 conn(0x7fe54c0cdbb0 msgr2=0x7fe54c15a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 shutdown_connections 2024-09-21T02:09:35.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:35.912+0000 7fe551e65640 1 -- 172.21.15.96:0/2042802230 wait complete. 2024-09-21T02:09:35.923 INFO:teuthology.orchestra.run.smithi096.stdout:60129542153 2024-09-21T02:09:35.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T02:09:35.924 INFO:teuthology.orchestra.run.smithi096.stdout:60129542154 2024-09-21T02:09:35.924 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T02:09:35.934 INFO:teuthology.orchestra.run.smithi096.stdout:60129542155 2024-09-21T02:09:35.934 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T02:09:35.936 INFO:teuthology.orchestra.run.smithi096.stdout:60129542154 2024-09-21T02:09:35.936 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T02:09:35.940 INFO:teuthology.orchestra.run.smithi096.stdout:60129542156 2024-09-21T02:09:35.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T02:09:35.945 INFO:teuthology.orchestra.run.smithi096.stdout:60129542155 2024-09-21T02:09:35.945 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T02:09:35.953 INFO:teuthology.orchestra.run.smithi096.stdout:60129542156 2024-09-21T02:09:35.953 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T02:09:36.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 -- 172.21.15.96:0/408796413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0cef70 msgr2=0x7f297c15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 --2- 172.21.15.96:0/408796413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0cef70 0x7f297c15c250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f297c0c6ef0 tx=0x7f297001c890 comp rx=0 tx=0).stop 2024-09-21T02:09:36.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 -- 172.21.15.96:0/2153815518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57815a660 msgr2=0x7ff5780cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 --2- 172.21.15.96:0/2153815518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57815a660 0x7ff5780cccb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff5780733c0 tx=0x7ff57001d7f0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 -- 172.21.15.96:0/408796413 shutdown_connections 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 --2- 172.21.15.96:0/408796413 >> v2:172.21.15.122:3300/0 conn(0x7f297c0704f0 0x7f297c0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 --2- 172.21.15.96:0/408796413 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 0x7f297c0ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 --2- 172.21.15.96:0/408796413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0cef70 0x7f297c15c250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 -- 172.21.15.96:0/408796413 >> 172.21.15.96:0/408796413 conn(0x7f297c06ce90 msgr2=0x7f297c06d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 -- 172.21.15.96:0/408796413 shutdown_connections 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 -- 172.21.15.96:0/2153815518 shutdown_connections 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 --2- 172.21.15.96:0/2153815518 >> v2:172.21.15.122:3300/0 conn(0x7ff5780cd1f0 0x7ff578073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 --2- 172.21.15.96:0/2153815518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57815a660 0x7ff5780cccb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 -- 172.21.15.96:0/2153815518 >> 172.21.15.96:0/2153815518 conn(0x7ff5780d03f0 msgr2=0x7ff5780d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f01299be640 1 -- 172.21.15.96:0/2209889538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c006320 msgr2=0x7f011c0edbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f01299be640 1 --2- 172.21.15.96:0/2209889538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c006320 0x7f011c0edbd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0110009900 tx=0x7f011001c920 comp rx=0 tx=0).stop 2024-09-21T02:09:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f01299be640 1 -- 172.21.15.96:0/2209889538 shutdown_connections 2024-09-21T02:09:36.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f01299be640 1 --2- 172.21.15.96:0/2209889538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c006320 0x7f011c0edbd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f01299be640 1 --2- 172.21.15.96:0/2209889538 >> v2:172.21.15.122:3300/0 conn(0x7f011c033bc0 0x7f011c0511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f01299be640 1 -- 172.21.15.96:0/2209889538 >> 172.21.15.96:0/2209889538 conn(0x7f011c01d9b0 msgr2=0x7f011c039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 -- 172.21.15.96:0/2153815518 shutdown_connections 2024-09-21T02:09:36.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7ff580081640 1 -- 172.21.15.96:0/2153815518 wait complete. 2024-09-21T02:09:36.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.132+0000 7f298161e640 1 -- 172.21.15.96:0/408796413 wait complete. 2024-09-21T02:09:36.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f01299be640 1 -- 172.21.15.96:0/2209889538 shutdown_connections 2024-09-21T02:09:36.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f298161e640 1 Processor -- start 2024-09-21T02:09:36.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f298161e640 1 -- start start 2024-09-21T02:09:36.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7ff580081640 1 Processor -- start 2024-09-21T02:09:36.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f01299be640 1 -- 172.21.15.96:0/2209889538 wait complete. 2024-09-21T02:09:36.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f01299be640 1 Processor -- start 2024-09-21T02:09:36.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f298161e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 0x7f297c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f298161e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 0x7f297c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7ff580081640 1 -- start start 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f298161e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cef70 0x7f297c1fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f297b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 0x7f297c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 0x7f297c151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59968/0 (socket says 172.21.15.96:59968) 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297bfff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 0x7f297c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297bfff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 0x7f297c151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34456/0 (socket says 172.21.15.96:34456) 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff5780b3f70 0x7ff5781fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 -- 172.21.15.96:0/1147120670 learned_addr learned my addr 172.21.15.96:0/1147120670 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5780cd1f0 0x7ff5781ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815a660 0x7ff578163870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.136+0000 7f01299be640 1 -- start start 2024-09-21T02:09:36.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f297c162950 con 0x7f297c0704f0 2024-09-21T02:09:36.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff578167de0 con 0x7ff5780cd1f0 2024-09-21T02:09:36.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1627d0 con 0x7f297c0cef70 2024-09-21T02:09:36.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f297c162ad0 con 0x7f297c0700e0 2024-09-21T02:09:36.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff578167c60 con 0x7ff57815a660 2024-09-21T02:09:36.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f2980e1d640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cef70 0x7f297c1fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff578167f60 con 0x7ff5780b3f70 2024-09-21T02:09:36.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57de1d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff5780b3f70 0x7ff5781fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57de1d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff5780b3f70 0x7ff5781fd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34462/0 (socket says 172.21.15.96:34462) 2024-09-21T02:09:36.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57de1d640 1 -- 172.21.15.96:0/2491588752 learned_addr learned my addr 172.21.15.96:0/2491588752 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57e61e640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815a660 0x7ff578163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5780cd1f0 0x7ff5781ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 -- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 msgr2=0x7f297c151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 0x7f297c151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 -- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cef70 msgr2=0x7f297c1fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cef70 0x7f297c1fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c072b30 con 0x7f297c0704f0 2024-09-21T02:09:36.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01299be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 0x7f011c0e5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f297b7fe640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 0x7f297c151ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f296c00c970 tx=0x7f296c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01299be640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c033bc0 0x7f011c053b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 -- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff5780b3f70 msgr2=0x7ff5781fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff5780b3f70 0x7ff5781fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 -- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815a660 msgr2=0x7ff578163870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815a660 0x7ff578163870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 -- 172.21.15.96:0/2491588752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5781f0340 con 0x7ff5780cd1f0 2024-09-21T02:09:36.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01299be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c0516f0 0x7f011c04a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 0x7f011c0e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01299be640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f011c03acd0 con 0x7f011c006320 2024-09-21T02:09:36.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01299be640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f011c03ab50 con 0x7f011c0516f0 2024-09-21T02:09:36.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01299be640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f011c03ae50 con 0x7f011c033bc0 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 0x7f011c0e5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59980/0 (socket says 172.21.15.96:59980) 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 -- 172.21.15.96:0/3859991519 learned_addr learned my addr 172.21.15.96:0/3859991519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f0123fff640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c033bc0 0x7f011c053b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 -- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c033bc0 msgr2=0x7f011c053b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c033bc0 0x7f011c053b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 -- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c0516f0 msgr2=0x7f011c04a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01291bd640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c0516f0 0x7f011c04a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff57d61c640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5780cd1f0 0x7ff5781ef4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff57815f170 tx=0x7ff57400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c0516f0 0x7f011c04a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 -- 172.21.15.96:0/3859991519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c1852c0 con 0x7f011c006320 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f01289bc640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 0x7f011c0e5020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f012406bc20 tx=0x7f0124076610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574004220 con 0x7ff5780cd1f0 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5740043c0 con 0x7ff5780cd1f0 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57400cb10 con 0x7ff5780cd1f0 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 -- 172.21.15.96:0/2491588752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5781f0620 con 0x7ff5780cd1f0 2024-09-21T02:09:36.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7ff580081640 1 -- 172.21.15.96:0/2491588752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5781f0be0 con 0x7ff5780cd1f0 2024-09-21T02:09:36.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c007bb0 con 0x7f297c0704f0 2024-09-21T02:09:36.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f296c007e40 con 0x7f297c0704f0 2024-09-21T02:09:36.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c005240 con 0x7f297c0704f0 2024-09-21T02:09:36.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c1f8c30 con 0x7f297c0704f0 2024-09-21T02:09:36.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.140+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c072da0 con 0x7f297c0704f0 2024-09-21T02:09:36.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f29627fc640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29540052a0 con 0x7f297c0704f0 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124063ab0 con 0x7f011c006320 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0124063d40 con 0x7f011c006320 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012406ade0 con 0x7f011c006320 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c0e56b0 con 0x7f011c006320 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c0d2590 con 0x7f011c006320 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff57400ccb0 con 0x7ff5780cd1f0 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7ff580081640 1 -- 172.21.15.96:0/2491588752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff578073430 con 0x7ff5780cd1f0 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7ff56e7fc640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff55c073280 0x7ff55c075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff574094ce0 con 0x7ff5780cd1f0 2024-09-21T02:09:36.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7ff57de1d640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff55c073280 0x7ff55c075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011c0d2810 con 0x7f011c006320 2024-09-21T02:09:36.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f296c0053e0 con 0x7f297c0704f0 2024-09-21T02:09:36.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f2978ff9640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f29680732d0 0x7f29680757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f296c094950 con 0x7f297c0704f0 2024-09-21T02:09:36.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7ff57de1d640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff55c073280 0x7ff55c075790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff570000f90 tx=0x7ff5700b0040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.144+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f012406af80 con 0x7f011c006320 2024-09-21T02:09:36.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.148+0000 7f297bfff640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f29680732d0 0x7f29680757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.148+0000 7f01217fa640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0108073230 0x7f0108075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.148+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f012410ff30 con 0x7f011c006320 2024-09-21T02:09:36.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.148+0000 7f0123fff640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0108073230 0x7f0108075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.148+0000 7f0123fff640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0108073230 0x7f0108075740 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0118000f70 tx=0x7f0118009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.148+0000 7f297bfff640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f29680732d0 0x7f29680757e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29700af000 tx=0x7f297001b0d0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.156+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f297c1f8c30 con 0x7f297c0704f0 2024-09-21T02:09:36.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.164+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f01240dad30 con 0x7f011c006320 2024-09-21T02:09:36.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff57405f350 con 0x7ff5780cd1f0 2024-09-21T02:09:36.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 -- 172.21.15.96:0/2695424884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 msgr2=0x7f46740362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 --2- 172.21.15.96:0/2695424884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740362a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46740e7280 tx=0x7f467001c800 comp rx=0 tx=0).stop 2024-09-21T02:09:36.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 -- 172.21.15.96:0/2695424884 shutdown_connections 2024-09-21T02:09:36.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 --2- 172.21.15.96:0/2695424884 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 0x7f4674034a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 --2- 172.21.15.96:0/2695424884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740362a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 -- 172.21.15.96:0/2695424884 >> 172.21.15.96:0/2695424884 conn(0x7f467401eb00 msgr2=0x7f467403a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f07b2beb640 1 -- 172.21.15.96:0/579164540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15ba70 msgr2=0x7f07ac071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f07b2beb640 1 --2- 172.21.15.96:0/579164540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15ba70 0x7f07ac071210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0798009900 tx=0x7f079801c850 comp rx=0 tx=0).stop 2024-09-21T02:09:36.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 -- 172.21.15.96:0/2695424884 shutdown_connections 2024-09-21T02:09:36.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f4681987640 1 -- 172.21.15.96:0/2695424884 wait complete. 2024-09-21T02:09:36.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f07b2beb640 1 -- 172.21.15.96:0/579164540 shutdown_connections 2024-09-21T02:09:36.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.168+0000 7f07b2beb640 1 --2- 172.21.15.96:0/579164540 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac15be80 0x7f07ac071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7f07b2beb640 1 --2- 172.21.15.96:0/579164540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15ba70 0x7f07ac071210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7f07b2beb640 1 -- 172.21.15.96:0/579164540 >> 172.21.15.96:0/579164540 conn(0x7f07ac0d1130 msgr2=0x7f07ac0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7fc94f59e640 1 -- 172.21.15.96:0/1812549376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc948006060 msgr2=0x7fc948006470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1812549376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc948006060 0x7fc948006470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ee70 tx=0x7fc94001c6e0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7f07b2beb640 1 -- 172.21.15.96:0/579164540 shutdown_connections 2024-09-21T02:09:36.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7f07b2beb640 1 -- 172.21.15.96:0/579164540 wait complete. 2024-09-21T02:09:36.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7f4681987640 1 Processor -- start 2024-09-21T02:09:36.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.172+0000 7f4681987640 1 -- start start 2024-09-21T02:09:36.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 Processor -- start 2024-09-21T02:09:36.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 -- start start 2024-09-21T02:09:36.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f4681987640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 0x7f46740d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f4681987640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f4681987640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46740e3d20 0x7f46740e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f4681987640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46740f4a10 con 0x7f4674035e90 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60014/0 (socket says 172.21.15.96:60014) 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 -- 172.21.15.96:0/4042611103 learned_addr learned my addr 172.21.15.96:0/4042611103 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467affd640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 0x7f46740d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7fc94f59e640 1 -- 172.21.15.96:0/1812549376 shutdown_connections 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1812549376 >> v2:172.21.15.122:3300/0 conn(0x7fc9480069b0 0x7fc948006dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1812549376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc948006060 0x7fc948006470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7fc94f59e640 1 -- 172.21.15.96:0/1812549376 >> 172.21.15.96:0/1812549376 conn(0x7fc94801de00 msgr2=0x7fc948039ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467b7fe640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46740e3d20 0x7f46740e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46740f4890 con 0x7f46740e3d20 2024-09-21T02:09:36.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f46740f4b90 con 0x7f4674034640 2024-09-21T02:09:36.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7fc94f59e640 1 -- 172.21.15.96:0/1812549376 shutdown_connections 2024-09-21T02:09:36.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7fc94f59e640 1 -- 172.21.15.96:0/1812549376 wait complete. 2024-09-21T02:09:36.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac152550 0x7f07ac0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 -- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 msgr2=0x7f46740d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07ac15ba70 0x7f07ac153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 0x7f46740d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 -- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46740e3d20 msgr2=0x7f46740e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46740e3d20 0x7f46740e1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46740d4b20 con 0x7f4674035e90 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467affd640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 0x7f46740d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467b7fe640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46740e3d20 0x7f46740e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15be80 0x7f07ac153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac168a00 con 0x7f07ac15be80 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b0987640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac152550 0x7f07ac0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac168880 con 0x7f07ac15ba70 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b0987640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac152550 0x7f07ac0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34516/0 (socket says 172.21.15.96:34516) 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b0987640 1 -- 172.21.15.96:0/59633528 learned_addr learned my addr 172.21.15.96:0/59633528 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07abfff640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07ac15ba70 0x7f07ac153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b1188640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15be80 0x7f07ac153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f07b2beb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f07ac168b80 con 0x7f07ac152550 2024-09-21T02:09:36.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.176+0000 7f467a7fc640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740e1790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c400 tx=0x7f466c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b1188640 1 -- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac152550 msgr2=0x7f07ac0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b1188640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac152550 0x7f07ac0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b1188640 1 -- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07ac15ba70 msgr2=0x7f07ac153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b1188640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07ac15ba70 0x7f07ac153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b1188640 1 -- 172.21.15.96:0/59633528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac157e80 con 0x7f07ac15be80 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b1188640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15be80 0x7f07ac153550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e990 tx=0x7f07a400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c011790 con 0x7f4674035e90 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f466c017070 con 0x7f4674035e90 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c011cc0 con 0x7f4674035e90 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4674033e00 con 0x7f4674035e90 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46740d4df0 con 0x7f4674035e90 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400cd60 con 0x7f07ac15be80 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07a4004b30 con 0x7f07ac15be80 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4005020 con 0x7f07ac15be80 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac158160 con 0x7f07ac15be80 2024-09-21T02:09:36.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac158720 con 0x7f07ac15be80 2024-09-21T02:09:36.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7fc94f59e640 1 Processor -- start 2024-09-21T02:09:36.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7fc94f59e640 1 -- start start 2024-09-21T02:09:36.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078c0052a0 con 0x7f07ac15be80 2024-09-21T02:09:36.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7fc94f59e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc948006060 0x7fc94817afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.180+0000 7fc94f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 0x7fc9480d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 0x7fc9480d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 0x7fc9480d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60036/0 (socket says 172.21.15.96:60036) 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94e59c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc948006060 0x7fc94817afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94e59c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc948006060 0x7fc94817afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34532/0 (socket says 172.21.15.96:34532) 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94f59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948034f60 0x7fc948051d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 -- 172.21.15.96:0/1960237246 learned_addr learned my addr 172.21.15.96:0/1960237246 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9480ed370 con 0x7fc9480069b0 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc948004d60 con 0x7fc948034f60 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc9480ed4f0 con 0x7fc948006060 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94ed9d640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948034f60 0x7fc948051d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 -- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc948006060 msgr2=0x7fc94817afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc948006060 0x7fc94817afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 -- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948034f60 msgr2=0x7fc948051d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948034f60 0x7fc948051d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9480d5150 con 0x7fc9480069b0 2024-09-21T02:09:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7fc94dd9b640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 0x7fc9480d1fc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc94001ad90 tx=0x7fc9400a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f07a40051c0 con 0x7f07ac15be80 2024-09-21T02:09:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7f07a97fa640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f07900732d0 0x7f07900757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f07a40953c0 con 0x7f07ac15be80 2024-09-21T02:09:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.184+0000 7f07b0987640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f07900732d0 0x7f07900757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.188+0000 7f07b0987640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f07900732d0 0x7f07900757e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07a00036c0 tx=0x7f07a0000bd0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.188+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f07a405fa30 con 0x7f07ac15be80 2024-09-21T02:09:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.188+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400ac070 con 0x7fc9480069b0 2024-09-21T02:09:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.188+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9400054b0 con 0x7fc9480069b0 2024-09-21T02:09:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.188+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940004290 con 0x7fc9480069b0 2024-09-21T02:09:36.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94817b510 con 0x7fc9480069b0 2024-09-21T02:09:36.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9480e0230 con 0x7fc9480069b0 2024-09-21T02:09:36.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c0052a0 con 0x7f4674035e90 2024-09-21T02:09:36.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f466c012420 con 0x7f4674035e90 2024-09-21T02:09:36.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7f4663fff640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f46680732d0 0x7f46680757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f466c093ac0 con 0x7f4674035e90 2024-09-21T02:09:36.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7f467affd640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f46680732d0 0x7f46680757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9480e1120 con 0x7fc9480069b0 2024-09-21T02:09:36.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.192+0000 7f467affd640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f46680732d0 0x7f46680757e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46700ae000 tx=0x7f467001b040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.196+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc9400af070 con 0x7fc9480069b0 2024-09-21T02:09:36.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.196+0000 7fc93effd640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc938077760 0x7fc938079c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.196+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc94012f250 con 0x7fc9480069b0 2024-09-21T02:09:36.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.196+0000 7fc94e59c640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc938077760 0x7fc938079c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.196+0000 7fc94e59c640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc938077760 0x7fc938079c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc944007fd0 tx=0x7fc944009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.204+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f466c020020 con 0x7f4674035e90 2024-09-21T02:09:36.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.212+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc9400163f0 con 0x7fc9480069b0 2024-09-21T02:09:36.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 -- 172.21.15.96:0/1324352481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 msgr2=0x7f54c4155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 --2- 172.21.15.96:0/1324352481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 0x7f54c4155370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009bc0 tx=0x7f54bc01caa0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 -- 172.21.15.96:0/1324352481 shutdown_connections 2024-09-21T02:09:36.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 --2- 172.21.15.96:0/1324352481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 0x7f54c4155370 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 -- 172.21.15.96:0/1324352481 >> 172.21.15.96:0/1324352481 conn(0x7f54c40b1a30 msgr2=0x7f54c40b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 -- 172.21.15.96:0/1324352481 shutdown_connections 2024-09-21T02:09:36.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.224+0000 7f54cadbd640 1 -- 172.21.15.96:0/1324352481 wait complete. 2024-09-21T02:09:36.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7f54cadbd640 1 Processor -- start 2024-09-21T02:09:36.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 -- 172.21.15.96:0/2756065574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e1670 msgr2=0x7fd0dc034d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/2756065574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e1670 0x7fd0dc034d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0e0580 tx=0x7fd0d001c850 comp rx=0 tx=0).stop 2024-09-21T02:09:36.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7f54cadbd640 1 -- start start 2024-09-21T02:09:36.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 -- 172.21.15.96:0/2756065574 shutdown_connections 2024-09-21T02:09:36.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/2756065574 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e0fa0 0x7fd0dc035270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/2756065574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e1670 0x7fd0dc034d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 -- 172.21.15.96:0/2756065574 >> 172.21.15.96:0/2756065574 conn(0x7fd0dc01eaf0 msgr2=0x7fd0dc03a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 -- 172.21.15.96:0/2756065574 shutdown_connections 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7f54cadbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 0x7f54c41486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7f54cadbd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c406f610 0x7f54c4154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54cadbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 0x7f54c41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 0x7f54c41486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 0x7f54c41486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60044/0 (socket says 172.21.15.96:60044) 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 -- 172.21.15.96:0/2899513735 learned_addr learned my addr 172.21.15.96:0/2899513735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4073000 con 0x7f54c406dea0 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4072e80 con 0x7f54c4154f60 2024-09-21T02:09:36.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f54c4073180 con 0x7f54c406f610 2024-09-21T02:09:36.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.228+0000 7fd0e8f09640 1 -- 172.21.15.96:0/2756065574 wait complete. 2024-09-21T02:09:36.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 Processor -- start 2024-09-21T02:09:36.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c95ba640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c406f610 0x7f54c4154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 -- start start 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54ca5bc640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 0x7f54c41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 -- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c406f610 msgr2=0x7f54c4154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c406f610 0x7f54c4154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 -- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 msgr2=0x7f54c41548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 0x7f54c41548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c41526e0 con 0x7f54c406dea0 2024-09-21T02:09:36.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54c9dbb640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 0x7f54c41486f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54c000c450 tx=0x7f54c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 0x7fd0dc04e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e1670 0x7fd0dc0d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 0x7fd0dc0debc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 0x7fd0dc0debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 0x7fd0dc04e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 0x7fd0dc0debc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60058/0 (socket says 172.21.15.96:60058) 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e2ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e1670 0x7fd0dc0d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 0x7fd0dc04e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.96:56684/0 (socket says 172.21.15.96:56684) 2024-09-21T02:09:36.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e2ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e1670 0x7fd0dc0d63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:34556/0 (socket says 172.21.15.96:34556) 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e3fff640 1 -- 172.21.15.96:0/4269340546 learned_addr learned my addr 172.21.15.96:0/4269340546 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc006bc0 con 0x7fd0dc0e1a80 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc006a40 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc006d40 con 0x7fd0dc0e1670 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e37fe640 1 -- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e1670 msgr2=0x7fd0dc0d63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e37fe640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e1670 0x7fd0dc0d63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e37fe640 1 -- 172.21.15.96:0/4269340546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 msgr2=0x7fd0dc0debc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e37fe640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 0x7fd0dc0debc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e37fe640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc032510 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e3fff640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 0x7fd0dc0debc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:36.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.232+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0016020 con 0x7f54c406dea0 2024-09-21T02:09:36.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e37fe640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 0x7fd0dc04e3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0532b0 tx=0x7fd0d00a65b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c40b5150 con 0x7f54c406dea0 2024-09-21T02:09:36.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c40b5710 con 0x7f54c406dea0 2024-09-21T02:09:36.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c0012920 con 0x7f54c406dea0 2024-09-21T02:09:36.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0012d50 con 0x7f54c406dea0 2024-09-21T02:09:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00bb020 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d0003930 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0003d80 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc0327f0 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc04d5d0 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54c0019020 con 0x7f54c406dea0 2024-09-21T02:09:36.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c40052a0 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a80052a0 con 0x7f54c406dea0 2024-09-21T02:09:36.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.236+0000 7f54ba7fc640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54b00733f0 0x7f54b0075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f54c00940e0 con 0x7f54c406dea0 2024-09-21T02:09:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7f54c95ba640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54b00733f0 0x7f54b0075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd0d00be020 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7fd0cffff640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd0c80732d0 0x7fd0c80757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd0d012f010 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7fd0e2ffd640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd0c80732d0 0x7fd0c80757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:36.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.240+0000 7fd0e2ffd640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd0c80732d0 0x7fd0c80757e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0346d0 tx=0x7fd0d8009060 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.244+0000 7f54c95ba640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54b00733f0 0x7f54b0075900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54c414bf30 tx=0x7f54b4009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:36.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.248+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54c0020030 con 0x7f54c406dea0 2024-09-21T02:09:36.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.256+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd0d00c5030 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.476+0000 7ff580081640 1 -- 172.21.15.96:0/2491588752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff578051000 con 0x7ff5780cd1f0 2024-09-21T02:09:36.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.476+0000 7ff56e7fc640 1 -- 172.21.15.96:0/2491588752 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff574080e20 con 0x7ff5780cd1f0 2024-09-21T02:09:36.485 INFO:teuthology.orchestra.run.smithi096.stdout:60129542152 2024-09-21T02:09:36.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 -- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff55c073280 msgr2=0x7ff55c075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff55c073280 0x7ff55c075790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff570000f90 tx=0x7ff5700b0040 comp rx=0 tx=0).stop 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 -- 172.21.15.96:0/2491588752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5780cd1f0 msgr2=0x7ff5781ef4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5780cd1f0 0x7ff5781ef4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff57815f170 tx=0x7ff57400eb50 comp rx=0 tx=0).stop 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 -- 172.21.15.96:0/2491588752 shutdown_connections 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff55c073280 0x7ff55c075790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815a660 0x7ff578163870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5780cd1f0 0x7ff5781ef4b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 --2- 172.21.15.96:0/2491588752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff5780b3f70 0x7ff5781fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 -- 172.21.15.96:0/2491588752 >> 172.21.15.96:0/2491588752 conn(0x7ff5780d03f0 msgr2=0x7ff5780d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 -- 172.21.15.96:0/2491588752 shutdown_connections 2024-09-21T02:09:36.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.488+0000 7ff55bfff640 1 -- 172.21.15.96:0/2491588752 wait complete. 2024-09-21T02:09:36.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.504+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f011c0511b0 con 0x7f011c006320 2024-09-21T02:09:36.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.504+0000 7f01217fa640 1 -- 172.21.15.96:0/3859991519 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01240da630 con 0x7f011c006320 2024-09-21T02:09:36.508 INFO:teuthology.orchestra.run.smithi096.stdout:60129542153 2024-09-21T02:09:36.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0108073230 msgr2=0x7f0108075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0108073230 0x7f0108075740 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0118000f70 tx=0x7f0118009040 comp rx=0 tx=0).stop 2024-09-21T02:09:36.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 msgr2=0x7f011c0e5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 0x7f011c0e5020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f012406bc20 tx=0x7f0124076610 comp rx=0 tx=0).stop 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 shutdown_connections 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0108073230 0x7f0108075740 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c0516f0 0x7f011c04a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f011c033bc0 0x7f011c053b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 --2- 172.21.15.96:0/3859991519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f011c006320 0x7f011c0e5020 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 >> 172.21.15.96:0/3859991519 conn(0x7f011c01d9b0 msgr2=0x7f011c0e08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 shutdown_connections 2024-09-21T02:09:36.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.508+0000 7f01299be640 1 -- 172.21.15.96:0/3859991519 wait complete. 2024-09-21T02:09:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.512+0000 7f29627fc640 1 -- 172.21.15.96:0/1147120670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2954005590 con 0x7f297c0704f0 2024-09-21T02:09:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.512+0000 7f2978ff9640 1 -- 172.21.15.96:0/1147120670 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f296c014030 con 0x7f297c0704f0 2024-09-21T02:09:36.516 INFO:teuthology.orchestra.run.smithi096.stdout:60129542152 2024-09-21T02:09:36.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542152 for osd.0 2024-09-21T02:09:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f29680732d0 msgr2=0x7f29680757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f29680732d0 0x7f29680757e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29700af000 tx=0x7f297001b0d0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 msgr2=0x7f297c151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 0x7f297c151ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f296c00c970 tx=0x7f296c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 shutdown_connections 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f29680732d0 0x7f29680757e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cef70 0x7f297c1fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297c0704f0 0x7f297c151ea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 --2- 172.21.15.96:0/1147120670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f297c0700e0 0x7f297c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 >> 172.21.15.96:0/1147120670 conn(0x7f297c06ce90 msgr2=0x7f297c0cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 shutdown_connections 2024-09-21T02:09:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.516+0000 7f298161e640 1 -- 172.21.15.96:0/1147120670 wait complete. 2024-09-21T02:09:36.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.532+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f465c005590 con 0x7f4674035e90 2024-09-21T02:09:36.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4663fff640 1 -- 172.21.15.96:0/4042611103 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f466c05e130 con 0x7f4674035e90 2024-09-21T02:09:36.539 INFO:teuthology.orchestra.run.smithi096.stdout:60129542153 2024-09-21T02:09:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f078c005860 con 0x7f07ac15be80 2024-09-21T02:09:36.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542153 for osd.5 2024-09-21T02:09:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f07a97fa640 1 -- 172.21.15.96:0/59633528 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f07a4064460 con 0x7f07ac15be80 2024-09-21T02:09:36.542 INFO:teuthology.orchestra.run.smithi096.stdout:60129542154 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f46680732d0 msgr2=0x7f46680757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f46680732d0 0x7f46680757e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46700ae000 tx=0x7f467001b040 comp rx=0 tx=0).stop 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 msgr2=0x7f46740e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740e1790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c400 tx=0x7f466c00c8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 shutdown_connections 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f46680732d0 0x7f46680757e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46740e3d20 0x7f46740e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4674035e90 0x7f46740e1790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 --2- 172.21.15.96:0/4042611103 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4674034640 0x7f46740d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 >> 172.21.15.96:0/4042611103 conn(0x7f467401eb00 msgr2=0x7f4674037b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 shutdown_connections 2024-09-21T02:09:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.536+0000 7f4681987640 1 -- 172.21.15.96:0/4042611103 wait complete. 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f07900732d0 msgr2=0x7f07900757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f07900732d0 0x7f07900757e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07a00036c0 tx=0x7f07a0000bd0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15be80 msgr2=0x7f07ac153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15be80 0x7f07ac153550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e990 tx=0x7f07a400ee50 comp rx=0 tx=0).stop 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 shutdown_connections 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f07900732d0 0x7f07900757e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ac15be80 0x7f07ac153550 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07ac15ba70 0x7f07ac153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 --2- 172.21.15.96:0/59633528 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f07ac152550 0x7f07ac0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 >> 172.21.15.96:0/59633528 conn(0x7f07ac0d1130 msgr2=0x7f07ac0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 shutdown_connections 2024-09-21T02:09:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.540+0000 7f07b2beb640 1 -- 172.21.15.96:0/59633528 wait complete. 2024-09-21T02:09:36.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542152 for osd.3 2024-09-21T02:09:36.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.548+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc9480e1120 con 0x7fc9480069b0 2024-09-21T02:09:36.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.548+0000 7fc93effd640 1 -- 172.21.15.96:0/1960237246 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9400f98d0 con 0x7fc9480069b0 2024-09-21T02:09:36.556 INFO:teuthology.orchestra.run.smithi096.stdout:60129542151 2024-09-21T02:09:36.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc938077760 msgr2=0x7fc938079c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc938077760 0x7fc938079c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc944007fd0 tx=0x7fc944009040 comp rx=0 tx=0).stop 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 msgr2=0x7fc9480d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 0x7fc9480d1fc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc94001ad90 tx=0x7fc9400a65d0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 shutdown_connections 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc938077760 0x7fc938079c70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948034f60 0x7fc948051d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9480069b0 0x7fc9480d1fc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 --2- 172.21.15.96:0/1960237246 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc948006060 0x7fc94817afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 >> 172.21.15.96:0/1960237246 conn(0x7fc94801de00 msgr2=0x7fc9480de030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 shutdown_connections 2024-09-21T02:09:36.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.552+0000 7fc94f59e640 1 -- 172.21.15.96:0/1960237246 wait complete. 2024-09-21T02:09:36.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542154 for osd.2 2024-09-21T02:09:36.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f54a8005590 con 0x7f54c406dea0 2024-09-21T02:09:36.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54ba7fc640 1 -- 172.21.15.96:0/2899513735 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54c005e7e0 con 0x7f54c406dea0 2024-09-21T02:09:36.572 INFO:teuthology.orchestra.run.smithi096.stdout:60129542154 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54b00733f0 msgr2=0x7f54b0075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54b00733f0 0x7f54b0075900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54c414bf30 tx=0x7f54b4009040 comp rx=0 tx=0).stop 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 msgr2=0x7f54c41486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 0x7f54c41486f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54c000c450 tx=0x7f54c000c910 comp rx=0 tx=0).stop 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 shutdown_connections 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54b00733f0 0x7f54b0075900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c4154f60 0x7f54c41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c406f610 0x7f54c4154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 --2- 172.21.15.96:0/2899513735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c406dea0 0x7f54c41486f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 >> 172.21.15.96:0/2899513735 conn(0x7f54c40b1a30 msgr2=0x7f54c406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 shutdown_connections 2024-09-21T02:09:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.568+0000 7f54cadbd640 1 -- 172.21.15.96:0/2899513735 wait complete. 2024-09-21T02:09:36.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542153 for osd.1 2024-09-21T02:09:36.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542151 for osd.6 2024-09-21T02:09:36.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.584+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd0c4005590 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.584+0000 7fd0cffff640 1 -- 172.21.15.96:0/4269340546 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0d00164d0 con 0x7fd0dc0e0fa0 2024-09-21T02:09:36.589 INFO:teuthology.orchestra.run.smithi096.stdout:60129542153 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd0c80732d0 msgr2=0x7fd0c80757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd0c80732d0 0x7fd0c80757e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0346d0 tx=0x7fd0d8009060 comp rx=0 tx=0).stop 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 msgr2=0x7fd0dc04e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 0x7fd0dc04e3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0532b0 tx=0x7fd0d00a65b0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 shutdown_connections 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd0c80732d0 0x7fd0c80757e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0dc0e1a80 0x7fd0dc0debc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0dc0e1670 0x7fd0dc0d63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 --2- 172.21.15.96:0/4269340546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0dc0e0fa0 0x7fd0dc04e3e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 >> 172.21.15.96:0/4269340546 conn(0x7fd0dc01eaf0 msgr2=0x7fd0dc03a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 shutdown_connections 2024-09-21T02:09:36.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:36.588+0000 7fd0e8f09640 1 -- 172.21.15.96:0/4269340546 wait complete. 2024-09-21T02:09:36.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542154 for osd.4 2024-09-21T02:09:36.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542153 for osd.7 2024-09-21T02:09:37.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T02:09:37.543 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T02:09:37.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T02:09:37.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T02:09:37.576 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T02:09:37.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T02:09:37.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T02:09:37.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T02:09:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 -- 172.21.15.96:0/1836284246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0700e0 msgr2=0x7fe3bc0704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 --2- 172.21.15.96:0/1836284246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc0704f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0ad840 tx=0x7fe3b401c740 comp rx=0 tx=0).stop 2024-09-21T02:09:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 -- 172.21.15.96:0/1836284246 shutdown_connections 2024-09-21T02:09:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 --2- 172.21.15.96:0/1836284246 >> v2:172.21.15.96:3300/0 conn(0x7fe3bc070a30 0x7fe3bc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 --2- 172.21.15.96:0/1836284246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc0704f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 -- 172.21.15.96:0/1836284246 >> 172.21.15.96:0/1836284246 conn(0x7fe3bc0d0ad0 msgr2=0x7fe3bc06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 -- 172.21.15.96:0/1836284246 shutdown_connections 2024-09-21T02:09:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.728+0000 7fe3c348b640 1 -- 172.21.15.96:0/1836284246 wait complete. 2024-09-21T02:09:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.732+0000 7fe3c348b640 1 Processor -- start 2024-09-21T02:09:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.732+0000 7fe3c348b640 1 -- start start 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3bc070a30 0x7fe3bc14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0b4df0 0x7fe3bc150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc15cdc0 con 0x7fe3bc0700e0 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc15cc40 con 0x7fe3bc0b4df0 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc15cf40 con 0x7fe3bc070a30 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c1227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c1227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42588/0 (socket says 172.21.15.96:42588) 2024-09-21T02:09:37.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c1227640 1 -- 172.21.15.96:0/3880206002 learned_addr learned my addr 172.21.15.96:0/3880206002 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3bc070a30 0x7fe3bc14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c1a28640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0b4df0 0x7fe3bc150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 -- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0b4df0 msgr2=0x7fe3bc150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0b4df0 0x7fe3bc150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 -- 172.21.15.96:0/3880206002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 msgr2=0x7fe3bc149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 -- 172.21.15.96:0/3880206002 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc069650 con 0x7fe3bc070a30 2024-09-21T02:09:37.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c1227640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:37.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c0a26640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3bc070a30 0x7fe3bc14fc10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40ac000 tx=0x7fe3b4005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40bc070 con 0x7fe3bc070a30 2024-09-21T02:09:37.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 -- 172.21.15.96:0/3880206002 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc1528e0 con 0x7fe3bc070a30 2024-09-21T02:09:37.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3c348b640 1 -- 172.21.15.96:0/3880206002 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc151490 con 0x7fe3bc070a30 2024-09-21T02:09:37.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.736+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3b40042f0 con 0x7fe3bc070a30 2024-09-21T02:09:37.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.740+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40b2040 con 0x7fe3bc070a30 2024-09-21T02:09:37.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.740+0000 7fe3c348b640 1 -- 172.21.15.96:0/3880206002 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c0052a0 con 0x7fe3bc070a30 2024-09-21T02:09:37.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.740+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3b40bf070 con 0x7fe3bc070a30 2024-09-21T02:09:37.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.740+0000 7fe3b1ffb640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe3a0073200 0x7fe3a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.744+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe3b412e050 con 0x7fe3bc070a30 2024-09-21T02:09:37.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.744+0000 7fe3c1227640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe3a0073200 0x7fe3a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.744+0000 7fe3c1227640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe3a0073200 0x7fe3a0075710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000f70 tx=0x7fe3b8009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.756+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe3b40166b0 con 0x7fe3bc070a30 2024-09-21T02:09:37.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/1957899256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc0365d0 msgr2=0x7fd4dc0369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 --2- 172.21.15.96:0/1957899256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc0365d0 0x7fd4dc0369e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4e406c350 tx=0x7fd4e4098170 comp rx=0 tx=0).stop 2024-09-21T02:09:37.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/1957899256 shutdown_connections 2024-09-21T02:09:37.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 --2- 172.21.15.96:0/1957899256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc0365d0 0x7fd4dc0369e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 --2- 172.21.15.96:0/1957899256 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4dc054dd0 0x7fd4dc0551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/1957899256 >> 172.21.15.96:0/1957899256 conn(0x7fd4dc01eaf0 msgr2=0x7fd4dc03a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/1957899256 shutdown_connections 2024-09-21T02:09:37.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/1957899256 wait complete. 2024-09-21T02:09:37.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7f17e4f1b640 1 -- 172.21.15.96:0/1674468143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 msgr2=0x7f17d80e0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/1674468143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d80e0dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f17e0077280 tx=0x7f17e0098330 comp rx=0 tx=0).stop 2024-09-21T02:09:37.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7f17e4f1b640 1 -- 172.21.15.96:0/1674468143 shutdown_connections 2024-09-21T02:09:37.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/1674468143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d80e0dd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/1674468143 >> v2:172.21.15.122:3300/0 conn(0x7f17d8035440 0x7f17d80e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.792+0000 7f17e4f1b640 1 -- 172.21.15.96:0/1674468143 >> 172.21.15.96:0/1674468143 conn(0x7f17d801eaf0 msgr2=0x7f17d803a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7f17e4f1b640 1 -- 172.21.15.96:0/1674468143 shutdown_connections 2024-09-21T02:09:37.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7f17e4f1b640 1 -- 172.21.15.96:0/1674468143 wait complete. 2024-09-21T02:09:37.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 Processor -- start 2024-09-21T02:09:37.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 -- start start 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4dc0365d0 0x7fd4dc033810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc053420 0x7fd4dc050700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 0x7fd4dc0e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc036090 con 0x7fd4dc054dd0 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc035f10 con 0x7fd4dc053420 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4eb3c6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd4dc036210 con 0x7fd4dc0365d0 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4e9963640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 0x7fd4dc0e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4e9963640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 0x7fd4dc0e0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42596/0 (socket says 172.21.15.96:42596) 2024-09-21T02:09:37.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4e9963640 1 -- 172.21.15.96:0/3434347383 learned_addr learned my addr 172.21.15.96:0/3434347383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4e9162640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4dc0365d0 0x7fd4dc033810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.796+0000 7fd4e8961640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc053420 0x7fd4dc050700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e9963640 1 -- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4dc0365d0 msgr2=0x7fd4dc033810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e9963640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4dc0365d0 0x7fd4dc033810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e9963640 1 -- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc053420 msgr2=0x7fd4dc050700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e9963640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc053420 0x7fd4dc050700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e9963640 1 -- 172.21.15.96:0/3434347383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc035220 con 0x7fd4dc054dd0 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e8961640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc053420 0x7fd4dc050700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:37.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4e9963640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 0x7fd4dc0e0e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd4e406c350 tx=0x7fd4e4128e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e412d040 con 0x7fd4dc054dd0 2024-09-21T02:09:37.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e41315e0 con 0x7fd4dc054dd0 2024-09-21T02:09:37.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/3434347383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc034180 con 0x7fd4dc054dd0 2024-09-21T02:09:37.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4131990 con 0x7fd4dc054dd0 2024-09-21T02:09:37.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4eb3c6640 1 -- 172.21.15.96:0/3434347383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc034740 con 0x7fd4dc054dd0 2024-09-21T02:09:37.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.800+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc0052a0 con 0x7fd4dc054dd0 2024-09-21T02:09:37.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.804+0000 7f17e4f1b640 1 Processor -- start 2024-09-21T02:09:37.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.804+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd4e4138030 con 0x7fd4dc054dd0 2024-09-21T02:09:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.804+0000 7fd4d9ffb640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd4c8073200 0x7fd4c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.804+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd4e41aa190 con 0x7fd4dc054dd0 2024-09-21T02:09:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.804+0000 7fd4e9162640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd4c8073200 0x7fd4c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.804+0000 7fd4e9162640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd4c8073200 0x7fd4c8075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4007fd0 tx=0x7fd4d4009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 -- start start 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17d80e1670 0x7f17d804fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d8052830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17d80d8600 0x7f17d817a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80ed650 con 0x7f17d80e1a80 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8034b00 con 0x7f17d80d8600 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17e4f1b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f17d80ed7d0 con 0x7f17d80e1670 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d8052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d8052830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42612/0 (socket says 172.21.15.96:42612) 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 -- 172.21.15.96:0/2546106718 learned_addr learned my addr 172.21.15.96:0/2546106718 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17dffff640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17d80d8600 0x7f17d817a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17df7fe640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17d80e1670 0x7f17d804fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 -- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17d80e1670 msgr2=0x7f17d804fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17d80e1670 0x7f17d804fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 -- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17d80d8600 msgr2=0x7f17d817a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17d80d8600 0x7f17d817a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 -- 172.21.15.96:0/2546106718 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d8052d70 con 0x7f17d80e1a80 2024-09-21T02:09:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17dffff640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17d80d8600 0x7f17d817a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.808+0000 7f17deffd640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d8052830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f17d400ee90 tx=0x7f17d4009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd4e413f020 con 0x7fd4dc054dd0 2024-09-21T02:09:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d400bbd0 con 0x7f17d80e1a80 2024-09-21T02:09:37.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d80505b0 con 0x7f17d80e1a80 2024-09-21T02:09:37.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d817a1c0 con 0x7f17d80e1a80 2024-09-21T02:09:37.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d40154d0 con 0x7f17d80e1a80 2024-09-21T02:09:37.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d400be60 con 0x7f17d80e1a80 2024-09-21T02:09:37.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.812+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f17d40158e0 con 0x7f17d80e1a80 2024-09-21T02:09:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.816+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c00052a0 con 0x7f17d80e1a80 2024-09-21T02:09:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.816+0000 7f17cbfff640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f17c4073200 0x7f17c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.816+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f17d4098b60 con 0x7f17d80e1a80 2024-09-21T02:09:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.816+0000 7f17df7fe640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f17c4073200 0x7f17c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.816+0000 7f17df7fe640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f17c4073200 0x7f17c4075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098e0 tx=0x7f17cc009060 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.820+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f17d4019030 con 0x7f17d80e1a80 2024-09-21T02:09:37.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485bbda640 1 -- 172.21.15.96:0/2437775334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 msgr2=0x7f4840003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485bbda640 1 --2- 172.21.15.96:0/2437775334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4840003e80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c590 tx=0x7f484c01d230 comp rx=0 tx=0).stop 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485a177640 1 -- 172.21.15.96:0/2437775334 reap_dead start 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485bbda640 1 -- 172.21.15.96:0/2437775334 shutdown_connections 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485bbda640 1 -- 172.21.15.96:0/2437775334 >> 172.21.15.96:0/2437775334 conn(0x7f48540cfdc0 msgr2=0x7f48540d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485bbda640 1 -- 172.21.15.96:0/2437775334 shutdown_connections 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f182d709640 1 -- 172.21.15.96:0/3545517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18280d0770 msgr2=0x7f18280b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f182d709640 1 --2- 172.21.15.96:0/3545517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18280d0770 0x7f18280b3e90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f182806fb40 tx=0x7f181801c8b0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 -- 172.21.15.96:0/3545517601 shutdown_connections 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 --2- 172.21.15.96:0/3545517601 >> v2:172.21.15.122:3300/0 conn(0x7f182806efb0 0x7f182806f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 --2- 172.21.15.96:0/3545517601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1828071a00 0x7f1828071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 --2- 172.21.15.96:0/3545517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18280d0770 0x7f18280b3e90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 -- 172.21.15.96:0/3545517601 >> 172.21.15.96:0/3545517601 conn(0x7f1828073af0 msgr2=0x7f1828073f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.824+0000 7f485bbda640 1 -- 172.21.15.96:0/2437775334 wait complete. 2024-09-21T02:09:37.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 -- 172.21.15.96:0/3545517601 shutdown_connections 2024-09-21T02:09:37.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 -- 172.21.15.96:0/3545517601 wait complete. 2024-09-21T02:09:37.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f485bbda640 1 Processor -- start 2024-09-21T02:09:37.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f485bbda640 1 -- start start 2024-09-21T02:09:37.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.828+0000 7f182d709640 1 Processor -- start 2024-09-21T02:09:37.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 -- start start 2024-09-21T02:09:37.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485bbda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4854204620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485bbda640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4854208360 0x7f4854208770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485bbda640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48541532c0 0x7f48541536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4854204620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4854204620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42636/0 (socket says 172.21.15.96:42636) 2024-09-21T02:09:37.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f182806efb0 0x7f182813ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485bbda640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48540af420 con 0x7f4840003a10 2024-09-21T02:09:37.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485bbda640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f485406e8b0 con 0x7f4854208360 2024-09-21T02:09:37.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485bbda640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f48540af5a0 con 0x7f48541532c0 2024-09-21T02:09:37.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859175640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4854208360 0x7f4854208770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1828071a00 0x7f18280b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18280d0770 0x7f18280b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18280cf6f0 con 0x7f1828071a00 2024-09-21T02:09:37.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1828071540 con 0x7f182806efb0 2024-09-21T02:09:37.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182d709640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f18280cf870 con 0x7f18280d0770 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485a177640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48541532c0 0x7f48541536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 -- 172.21.15.96:0/3048049767 learned_addr learned my addr 172.21.15.96:0/3048049767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f485a177640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48541532c0 0x7f48541536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:59946/0 (socket says 172.21.15.96:59946) 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f1827fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f182806efb0 0x7f182813ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f182cf08640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18280d0770 0x7f18280b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f1827fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f182806efb0 0x7f182813ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.96:35378/0 (socket says 172.21.15.96:35378) 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f1827fff640 1 -- 172.21.15.96:0/444365229 learned_addr learned my addr 172.21.15.96:0/444365229 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1828071a00 0x7f18280b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 -- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48541532c0 msgr2=0x7f48541536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48541532c0 0x7f48541536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 -- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4854208360 msgr2=0x7f4854208770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4854208360 0x7f4854208770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 -- 172.21.15.96:0/3048049767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4854162420 con 0x7f4840003a10 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 -- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18280d0770 msgr2=0x7f18280b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18280d0770 0x7f18280b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 -- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f182806efb0 msgr2=0x7f182813ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f182806efb0 0x7f182813ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 -- 172.21.15.96:0/444365229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18281f4f40 con 0x7f1828071a00 2024-09-21T02:09:37.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f4859976640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4854204620 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f484c01b930 tx=0x7f484c006680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.832+0000 7f18277fe640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1828071a00 0x7f18280b46f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1828144f60 tx=0x7f181c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c017020 con 0x7f1828071a00 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f181c004320 con 0x7f1828071a00 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c009a20 con 0x7f1828071a00 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182813e330 con 0x7f1828071a00 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1828161f30 con 0x7f1828071a00 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c004db0 con 0x7f4840003a10 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f484c006870 con 0x7f4840003a10 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0039b0 con 0x7f4840003a10 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f485bbda640 1 -- 172.21.15.96:0/3048049767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4854162700 con 0x7f4840003a10 2024-09-21T02:09:37.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f485bbda640 1 -- 172.21.15.96:0/3048049767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4854207e60 con 0x7f4840003a10 2024-09-21T02:09:37.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.836+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48300052a0 con 0x7f4840003a10 2024-09-21T02:09:37.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 -- 172.21.15.96:0/1339172780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 msgr2=0x7f320c0d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 --2- 172.21.15.96:0/1339172780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c0d1690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3208009840 tx=0x7f320801c7b0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 -- 172.21.15.96:0/1339172780 shutdown_connections 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 --2- 172.21.15.96:0/1339172780 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c072e30 0x7f320c15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 --2- 172.21.15.96:0/1339172780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c0d1690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 -- 172.21.15.96:0/1339172780 >> 172.21.15.96:0/1339172780 conn(0x7f320c0d20a0 msgr2=0x7f320c0d24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 -- 172.21.15.96:0/1339172780 shutdown_connections 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 -- 172.21.15.96:0/1339172780 wait complete. 2024-09-21T02:09:37.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18080052a0 con 0x7f1828071a00 2024-09-21T02:09:37.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.840+0000 7f32132fd640 1 Processor -- start 2024-09-21T02:09:37.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 -- 172.21.15.96:0/2917006944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4840ada50 msgr2=0x7fc48415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 --2- 172.21.15.96:0/2917006944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4840ada50 0x7fc48415e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc474009930 tx=0x7fc47401c940 comp rx=0 tx=0).stop 2024-09-21T02:09:37.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 -- 172.21.15.96:0/3497847195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc15ef70 msgr2=0x7f19cc06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 --2- 172.21.15.96:0/3497847195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc15ef70 0x7f19cc06dea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19cc070a00 tx=0x7f19c801c5c0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 -- 172.21.15.96:0/3497847195 shutdown_connections 2024-09-21T02:09:37.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 --2- 172.21.15.96:0/3497847195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc15ef70 0x7f19cc06dea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 --2- 172.21.15.96:0/3497847195 >> v2:172.21.15.96:3300/0 conn(0x7f19cc155370 0x7f19cc15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 -- 172.21.15.96:0/3497847195 >> 172.21.15.96:0/3497847195 conn(0x7f19cc0b1a30 msgr2=0x7f19cc0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 -- 172.21.15.96:0/3497847195 shutdown_connections 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f19d2b99640 1 -- 172.21.15.96:0/3497847195 wait complete. 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f484c003b50 con 0x7f4840003a10 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f484a7fc640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4838073300 0x7f4838075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f484c12f400 con 0x7f4840003a10 2024-09-21T02:09:37.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 -- start start 2024-09-21T02:09:37.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f4859175640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4838073300 0x7f4838075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 -- 172.21.15.96:0/2917006944 shutdown_connections 2024-09-21T02:09:37.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 --2- 172.21.15.96:0/2917006944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4840ada50 0x7fc48415e4d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 -- 172.21.15.96:0/2917006944 >> 172.21.15.96:0/2917006944 conn(0x7fc4840d1130 msgr2=0x7fc4840d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 -- 172.21.15.96:0/2917006944 shutdown_connections 2024-09-21T02:09:37.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f4859175640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4838073300 0x7f4838075810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f485415a0f0 tx=0x7f4844009540 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c0cfaa0 0x7f320c0c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c204da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 0x7f320c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f3211099640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c0cfaa0 0x7f320c0c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f3211099640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c0cfaa0 0x7f320c0c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:59976/0 (socket says 172.21.15.96:59976) 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f3210898640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c204da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f3210898640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c204da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.96:35390/0 (socket says 172.21.15.96:35390) 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f321189a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 0x7f320c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f321189a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 0x7f320c155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42660/0 (socket says 172.21.15.96:42660) 2024-09-21T02:09:37.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f3211099640 1 -- 172.21.15.96:0/2948680638 learned_addr learned my addr 172.21.15.96:0/2948680638 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f320c06e120 con 0x7f320c070790 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f320c06dfa0 con 0x7f320c0d0880 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f320c06e2a0 con 0x7f320c0cfaa0 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f321189a640 1 -- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c0cfaa0 msgr2=0x7f320c0c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f321189a640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c0cfaa0 0x7f320c0c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f321189a640 1 -- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 msgr2=0x7f320c204da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f321189a640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c204da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f321189a640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c2052e0 con 0x7f320c070790 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.844+0000 7fc488beb640 1 -- 172.21.15.96:0/2917006944 wait complete. 2024-09-21T02:09:37.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc488beb640 1 Processor -- start 2024-09-21T02:09:37.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 Processor -- start 2024-09-21T02:09:37.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 -- start start 2024-09-21T02:09:37.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f321189a640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 0x7f320c155270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f320400efc0 tx=0x7f320400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc488beb640 1 -- start start 2024-09-21T02:09:37.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f484c0c0060 con 0x7f4840003a10 2024-09-21T02:09:37.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32040092e0 con 0x7f320c070790 2024-09-21T02:09:37.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc488beb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc48406db30 0x7fc48414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc488beb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc484071210 0x7fc484200d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc488beb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 0x7fc484201270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc48259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc48406db30 0x7fc48414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc48259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc48406db30 0x7fc48414efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.96:35394/0 (socket says 172.21.15.96:35394) 2024-09-21T02:09:37.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc481d9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc484071210 0x7fc484200d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7fc481d9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc484071210 0x7fc484200d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:59978/0 (socket says 172.21.15.96:59978) 2024-09-21T02:09:37.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f181c01a020 con 0x7f1828071a00 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 0x7fc484201270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 0x7fc484201270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42670/0 (socket says 172.21.15.96:42670) 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc48259c640 1 -- 172.21.15.96:0/2550769776 learned_addr learned my addr 172.21.15.96:0/2550769776 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc48416ce40 con 0x7fc4840ada50 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840ae0e0 con 0x7fc48406db30 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc48416cfc0 con 0x7fc484071210 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19cc06e3e0 0x7f19cc15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc155370 0x7f19cc15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19cc15ef70 0x7f19cc205e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc073810 con 0x7f19cc15ef70 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc073690 con 0x7f19cc155370 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2b99640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc16c210 con 0x7f19cc06e3e0 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d1b97640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19cc06e3e0 0x7f19cc15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d1b97640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19cc06e3e0 0x7f19cc15b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.96:59982/0 (socket says 172.21.15.96:59982) 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d1396640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc155370 0x7f19cc15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d1b97640 1 -- 172.21.15.96:0/747756044 learned_addr learned my addr 172.21.15.96:0/747756044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f19d2398640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19cc15ef70 0x7f19cc205e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d1396640 1 -- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19cc06e3e0 msgr2=0x7f19cc15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d1396640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19cc06e3e0 0x7f19cc15b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d1396640 1 -- 172.21.15.96:0/747756044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19cc15ef70 msgr2=0x7f19cc205e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d1396640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19cc15ef70 0x7f19cc205e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d1396640 1 -- 172.21.15.96:0/747756044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc15b9c0 con 0x7f19cc155370 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d1396640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc155370 0x7f19cc15bbc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f19c80ac000 tx=0x7f19c80a9990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80ad070 con 0x7f19cc155370 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc15a3e0 con 0x7f19cc155370 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc15a9a0 con 0x7f19cc155370 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19c8003600 con 0x7f19cc155370 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f1824ff9640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f18140732d0 0x7f18140757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f181c094840 con 0x7f1828071a00 2024-09-21T02:09:37.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c1557b0 con 0x7f320c070790 2024-09-21T02:09:37.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c0cc620 con 0x7f320c070790 2024-09-21T02:09:37.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.848+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3204004b60 con 0x7f320c070790 2024-09-21T02:09:37.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320400cb20 con 0x7f320c070790 2024-09-21T02:09:37.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f1827fff640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f18140732d0 0x7f18140757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003a50 con 0x7f19cc155370 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 -- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc484071210 msgr2=0x7fc484200d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc484071210 0x7fc484200d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 -- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc48406db30 msgr2=0x7fc48414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc48406db30 0x7fc48414efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48414db30 con 0x7fc4840ada50 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b40052a0 con 0x7f19cc155370 2024-09-21T02:09:37.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec0052a0 con 0x7f320c070790 2024-09-21T02:09:37.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7fc482d9d640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 0x7fc484201270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c700 tx=0x7fc47c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f1827fff640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f18140732d0 0x7f18140757e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1818009230 tx=0x7f181801b0f0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.852+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19c80b0070 con 0x7f19cc155370 2024-09-21T02:09:37.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.856+0000 7f19c27fc640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b0073230 0x7f19b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.856+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f19c812ed20 con 0x7f19cc155370 2024-09-21T02:09:37.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.856+0000 7f19d1b97640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b0073230 0x7f19b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.856+0000 7f19d1b97640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b0073230 0x7f19b0075740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19c400e250 tx=0x7f19c400b040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.864+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f181c021030 con 0x7f1828071a00 2024-09-21T02:09:37.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.864+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f320401b030 con 0x7f320c070790 2024-09-21T02:09:37.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.864+0000 7f3201ffb640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073230 0x7f31f0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7f3211099640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073230 0x7f31f0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.864+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c003b80 con 0x7fc4840ada50 2024-09-21T02:09:37.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48414de10 con 0x7fc4840ada50 2024-09-21T02:09:37.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48414bd90 con 0x7fc4840ada50 2024-09-21T02:09:37.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f320401ee00 con 0x7f320c070790 2024-09-21T02:09:37.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7f3211099640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073230 0x7f31f0075740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f320801a3e0 tx=0x7f320801aff0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47c003d20 con 0x7fc4840ada50 2024-09-21T02:09:37.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19cc15b9c0 con 0x7f19cc155370 2024-09-21T02:09:37.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48415e4d0 con 0x7fc4840ada50 2024-09-21T02:09:37.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.868+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c006da0 con 0x7fc4840ada50 2024-09-21T02:09:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.876+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3204022020 con 0x7f320c070790 2024-09-21T02:09:37.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.880+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc47c0050d0 con 0x7fc4840ada50 2024-09-21T02:09:37.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.880+0000 7fc472ffd640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc464073200 0x7fc464075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.880+0000 7fc48259c640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc464073200 0x7fc464075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.884+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc47c092fe0 con 0x7fc4840ada50 2024-09-21T02:09:37.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.884+0000 7fc48259c640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc464073200 0x7fc464075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc474009230 tx=0x7fc47401b180 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:37.892+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc47c020020 con 0x7fc4840ada50 2024-09-21T02:09:38.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.104+0000 7fe3c348b640 1 -- 172.21.15.96:0/3880206002 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe39c005860 con 0x7fe3bc070a30 2024-09-21T02:09:38.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.104+0000 7fe3b1ffb640 1 -- 172.21.15.96:0/3880206002 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe3b40f87c0 con 0x7fe3bc070a30 2024-09-21T02:09:38.113 INFO:teuthology.orchestra.run.smithi096.stdout:60129542154 2024-09-21T02:09:38.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 -- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe3a0073200 msgr2=0x7fe3a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe3a0073200 0x7fe3a0075710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000f70 tx=0x7fe3b8009040 comp rx=0 tx=0).stop 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 -- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3bc070a30 msgr2=0x7fe3bc14fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3bc070a30 0x7fe3bc14fc10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40ac000 tx=0x7fe3b4005a60 comp rx=0 tx=0).stop 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 -- 172.21.15.96:0/3880206002 shutdown_connections 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe3a0073200 0x7fe3a0075710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3bc0b4df0 0x7fe3bc150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3bc070a30 0x7fe3bc14fc10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3880206002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3bc0700e0 0x7fe3bc149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 -- 172.21.15.96:0/3880206002 >> 172.21.15.96:0/3880206002 conn(0x7fe3bc0d0ad0 msgr2=0x7fe3bc073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 -- 172.21.15.96:0/3880206002 shutdown_connections 2024-09-21T02:09:38.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.108+0000 7fe3a77fe640 1 -- 172.21.15.96:0/3880206002 wait complete. 2024-09-21T02:09:38.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.0 2024-09-21T02:09:38.141 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.140+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd4bc005860 con 0x7fd4dc054dd0 2024-09-21T02:09:38.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.140+0000 7fd4d9ffb640 1 -- 172.21.15.96:0/3434347383 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4e4092010 con 0x7fd4dc054dd0 2024-09-21T02:09:38.146 INFO:teuthology.orchestra.run.smithi096.stdout:60129542155 2024-09-21T02:09:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd4c8073200 msgr2=0x7fd4c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd4c8073200 0x7fd4c8075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4007fd0 tx=0x7fd4d4009040 comp rx=0 tx=0).stop 2024-09-21T02:09:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 msgr2=0x7fd4dc0e0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 0x7fd4dc0e0e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd4e406c350 tx=0x7fd4e4128e10 comp rx=0 tx=0).stop 2024-09-21T02:09:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 shutdown_connections 2024-09-21T02:09:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd4c8073200 0x7fd4c8075710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4dc054dd0 0x7fd4dc0e0e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4dc053420 0x7fd4dc050700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 --2- 172.21.15.96:0/3434347383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4dc0365d0 0x7fd4dc033810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 >> 172.21.15.96:0/3434347383 conn(0x7fd4dc01eaf0 msgr2=0x7fd4dc00bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 shutdown_connections 2024-09-21T02:09:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.144+0000 7fd4cf7fe640 1 -- 172.21.15.96:0/3434347383 wait complete. 2024-09-21T02:09:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.148+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f17c0005590 con 0x7f17d80e1a80 2024-09-21T02:09:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.148+0000 7f17cbfff640 1 -- 172.21.15.96:0/2546106718 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f17d4063270 con 0x7f17d80e1a80 2024-09-21T02:09:38.151 INFO:teuthology.orchestra.run.smithi096.stdout:60129542154 2024-09-21T02:09:38.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f17c4073200 msgr2=0x7f17c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f17c4073200 0x7f17c4075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098e0 tx=0x7f17cc009060 comp rx=0 tx=0).stop 2024-09-21T02:09:38.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 msgr2=0x7f17d8052830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d8052830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f17d400ee90 tx=0x7f17d4009df0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 shutdown_connections 2024-09-21T02:09:38.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f17c4073200 0x7f17c4075710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17d80d8600 0x7f17d817a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d80e1a80 0x7f17d8052830 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 --2- 172.21.15.96:0/2546106718 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17d80e1670 0x7f17d804fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 >> 172.21.15.96:0/2546106718 conn(0x7f17d801eaf0 msgr2=0x7f17d80db680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 shutdown_connections 2024-09-21T02:09:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.152+0000 7f17e4f1b640 1 -- 172.21.15.96:0/2546106718 wait complete. 2024-09-21T02:09:38.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.5 2024-09-21T02:09:38.173 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.3 2024-09-21T02:09:38.185 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.192+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1808005860 con 0x7f1828071a00 2024-09-21T02:09:38.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.192+0000 7f1824ff9640 1 -- 172.21.15.96:0/444365229 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f181c05ef40 con 0x7f1828071a00 2024-09-21T02:09:38.197 INFO:teuthology.orchestra.run.smithi096.stdout:60129542155 2024-09-21T02:09:38.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4830005860 con 0x7f4840003a10 2024-09-21T02:09:38.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f484a7fc640 1 -- 172.21.15.96:0/3048049767 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f484c016f90 con 0x7f4840003a10 2024-09-21T02:09:38.200 INFO:teuthology.orchestra.run.smithi096.stdout:60129542156 2024-09-21T02:09:38.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f18140732d0 msgr2=0x7f18140757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f18140732d0 0x7f18140757e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1818009230 tx=0x7f181801b0f0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1828071a00 msgr2=0x7f18280b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1828071a00 0x7f18280b46f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1828144f60 tx=0x7f181c00dbb0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 shutdown_connections 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f18140732d0 0x7f18140757e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18280d0770 0x7f18280b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1828071a00 0x7f18280b46f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 --2- 172.21.15.96:0/444365229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f182806efb0 0x7f182813ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 >> 172.21.15.96:0/444365229 conn(0x7f1828073af0 msgr2=0x7f1828140d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 shutdown_connections 2024-09-21T02:09:38.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.196+0000 7f182d709640 1 -- 172.21.15.96:0/444365229 wait complete. 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4838073300 msgr2=0x7f4838075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4838073300 0x7f4838075810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f485415a0f0 tx=0x7f4844009540 comp rx=0 tx=0).stop 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 msgr2=0x7f4854204620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4854204620 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f484c01b930 tx=0x7f484c006680 comp rx=0 tx=0).stop 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 shutdown_connections 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4838073300 0x7f4838075810 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48541532c0 0x7f48541536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4854208360 0x7f4854208770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 --2- 172.21.15.96:0/3048049767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840003a10 0x7f4854204620 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f484c01b930 tx=0x7f484c006680 comp rx=0 tx=0).stop 2024-09-21T02:09:38.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 >> 172.21.15.96:0/3048049767 conn(0x7f48540cfdc0 msgr2=0x7f48540d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 shutdown_connections 2024-09-21T02:09:38.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.200+0000 7f4837fff640 1 -- 172.21.15.96:0/3048049767 wait complete. 2024-09-21T02:09:38.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f19b4005860 con 0x7f19cc155370 2024-09-21T02:09:38.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19c27fc640 1 -- 172.21.15.96:0/747756044 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19c80b7070 con 0x7f19cc155370 2024-09-21T02:09:38.212 INFO:teuthology.orchestra.run.smithi096.stdout:60129542156 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b0073230 msgr2=0x7f19b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b0073230 0x7f19b0075740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19c400e250 tx=0x7f19c400b040 comp rx=0 tx=0).stop 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc155370 msgr2=0x7f19cc15bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc155370 0x7f19cc15bbc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f19c80ac000 tx=0x7f19c80a9990 comp rx=0 tx=0).stop 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 shutdown_connections 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b0073230 0x7f19b0075740 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19cc15ef70 0x7f19cc205e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19cc155370 0x7f19cc15bbc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 --2- 172.21.15.96:0/747756044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19cc06e3e0 0x7f19cc15b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.208+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 >> 172.21.15.96:0/747756044 conn(0x7f19cc0b1a30 msgr2=0x7f19cc154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.212+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 shutdown_connections 2024-09-21T02:09:38.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.212+0000 7f19d2b99640 1 -- 172.21.15.96:0/747756044 wait complete. 2024-09-21T02:09:38.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.212+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f31ec005860 con 0x7f320c070790 2024-09-21T02:09:38.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.216+0000 7f3201ffb640 1 -- 172.21.15.96:0/2948680638 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f320c2052e0 con 0x7f320c070790 2024-09-21T02:09:38.219 INFO:teuthology.orchestra.run.smithi096.stdout:60129542153 2024-09-21T02:09:38.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073230 msgr2=0x7f31f0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073230 0x7f31f0075740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f320801a3e0 tx=0x7f320801aff0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 msgr2=0x7f320c155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 0x7f320c155270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f320400efc0 tx=0x7f320400c490 comp rx=0 tx=0).stop 2024-09-21T02:09:38.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 shutdown_connections 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073230 0x7f31f0075740 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f320c070790 0x7f320c155270 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c0d0880 0x7f320c204da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 --2- 172.21.15.96:0/2948680638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f320c0cfaa0 0x7f320c0c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 >> 172.21.15.96:0/2948680638 conn(0x7f320c0d20a0 msgr2=0x7f320c072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 shutdown_connections 2024-09-21T02:09:38.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.220+0000 7f32132fd640 1 -- 172.21.15.96:0/2948680638 wait complete. 2024-09-21T02:09:38.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.1 2024-09-21T02:09:38.231 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.4 2024-09-21T02:09:38.235 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.2 2024-09-21T02:09:38.237 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.232+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc484152b60 con 0x7fc4840ada50 2024-09-21T02:09:38.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.232+0000 7fc472ffd640 1 -- 172.21.15.96:0/2550769776 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc47c05f400 con 0x7fc4840ada50 2024-09-21T02:09:38.238 INFO:teuthology.orchestra.run.smithi096.stdout:60129542155 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc464073200 msgr2=0x7fc464075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc464073200 0x7fc464075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc474009230 tx=0x7fc47401b180 comp rx=0 tx=0).stop 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 msgr2=0x7fc484201270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 0x7fc484201270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c700 tx=0x7fc47c00cbc0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 shutdown_connections 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc464073200 0x7fc464075710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4840ada50 0x7fc484201270 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc484071210 0x7fc484200d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 --2- 172.21.15.96:0/2550769776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc48406db30 0x7fc48414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 >> 172.21.15.96:0/2550769776 conn(0x7fc4840d1130 msgr2=0x7fc4840b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 shutdown_connections 2024-09-21T02:09:38.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.236+0000 7fc488beb640 1 -- 172.21.15.96:0/2550769776 wait complete. 2024-09-21T02:09:38.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.6 2024-09-21T02:09:38.247 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.7 2024-09-21T02:09:38.258 DEBUG:teuthology.parallel:result is None 2024-09-21T02:09:38.258 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:09:38.259 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 -- 172.21.15.96:0/2177648085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415fb60 msgr2=0x7f11740d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 --2- 172.21.15.96:0/2177648085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415fb60 0x7f11740d0460 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1174073ce0 tx=0x7f116c01cd80 comp rx=0 tx=0).stop 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 -- 172.21.15.96:0/2177648085 shutdown_connections 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 --2- 172.21.15.96:0/2177648085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415fb60 0x7f11740d0460 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 --2- 172.21.15.96:0/2177648085 >> v2:172.21.15.122:3300/0 conn(0x7f117415f750 0x7f11740cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 -- 172.21.15.96:0/2177648085 >> 172.21.15.96:0/2177648085 conn(0x7f117406fbe0 msgr2=0x7f117406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 -- 172.21.15.96:0/2177648085 shutdown_connections 2024-09-21T02:09:38.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.436+0000 7f117c44a640 1 -- 172.21.15.96:0/2177648085 wait complete. 2024-09-21T02:09:38.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 Processor -- start 2024-09-21T02:09:38.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- start start 2024-09-21T02:09:38.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f117415b310 0x7f117414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 0x7f117414c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f117415fb60 0x7f11741579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1174168700 con 0x7f117415f750 2024-09-21T02:09:38.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1174168580 con 0x7f117415fb60 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1174168880 con 0x7f117415b310 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 0x7f117414c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 0x7f117414c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42696/0 (socket says 172.21.15.96:42696) 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 -- 172.21.15.96:0/2920986162 learned_addr learned my addr 172.21.15.96:0/2920986162 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117a9e7640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f117415fb60 0x7f11741579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117a1e6640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f117415b310 0x7f117414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 -- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f117415b310 msgr2=0x7f117414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f117415b310 0x7f117414b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 -- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f117415fb60 msgr2=0x7f11741579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f117415fb60 0x7f11741579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 -- 172.21.15.96:0/2920986162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1174156910 con 0x7f117415f750 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117a9e7640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f117415fb60 0x7f11741579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:38.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f11799e5640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 0x7f117414c350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f116400c9f0 tx=0x7f116400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:38.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164007bb0 con 0x7f117415f750 2024-09-21T02:09:38.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1164007e40 con 0x7f117415f750 2024-09-21T02:09:38.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1174156bf0 con 0x7f117415f750 2024-09-21T02:09:38.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400f690 con 0x7f117415f750 2024-09-21T02:09:38.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11741571b0 con 0x7f117415f750 2024-09-21T02:09:38.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.440+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11740cebc0 con 0x7f117415f750 2024-09-21T02:09:38.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.444+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f116400f830 con 0x7f117415f750 2024-09-21T02:09:38.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.444+0000 7f116affd640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1158073230 0x7f1158075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.444+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1164094d90 con 0x7f117415f750 2024-09-21T02:09:38.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.444+0000 7f117a1e6640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1158073230 0x7f1158075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.444+0000 7f117a1e6640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1158073230 0x7f1158075740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f117000a750 tx=0x7f1170008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:38.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.448+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1164014030 con 0x7f117415f750 2024-09-21T02:09:38.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.604+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11740b4b80 con 0x7f1158073230 2024-09-21T02:09:38.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.620+0000 7f116affd640 1 -- 172.21.15.96:0/2920986162 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332520 (secure 0 0 0) 0x7f11740b4b80 con 0x7f1158073230 2024-09-21T02:09:38.627 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:38.629 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:09:38.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1158073230 msgr2=0x7f1158075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1158073230 0x7f1158075740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f117000a750 tx=0x7f1170008040 comp rx=0 tx=0).stop 2024-09-21T02:09:38.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 msgr2=0x7f117414c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 0x7f117414c350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f116400c9f0 tx=0x7f116400ceb0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 shutdown_connections 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1158073230 0x7f1158075740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f117415fb60 0x7f11741579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f117415f750 0x7f117414c350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 --2- 172.21.15.96:0/2920986162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f117415b310 0x7f117414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 >> 172.21.15.96:0/2920986162 conn(0x7f117406fbe0 msgr2=0x7f11740d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 shutdown_connections 2024-09-21T02:09:38.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.624+0000 7f117c44a640 1 -- 172.21.15.96:0/2920986162 wait complete. 2024-09-21T02:09:38.647 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-21T02:09:38.507684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223768,"kb_used_data":3368,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750952,"statfs":{"total":773094113280,"available":772864974848,"internally_reserved":0,"allocated":3448832,"data_stored":2054672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005904"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723570+0000","last_change":"2024-09-21T02:09:10.527965+0000","last_active":"2024-09-21T02:09:16.723570+0000","last_peered":"2024-09-21T02:09:16.723570+0000","last_clean":"2024-09-21T02:09:16.723570+0000","last_became_active":"2024-09-21T02:09:10.527508+0000","last_became_peered":"2024-09-21T02:09:10.527508+0000","last_unstale":"2024-09-21T02:09:16.723570+0000","last_undegraded":"2024-09-21T02:09:16.723570+0000","last_fullsized":"2024-09-21T02:09:16.723570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:35.986615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590616+0000","last_change":"2024-09-21T02:09:11.546557+0000","last_active":"2024-09-21T02:09:16.590616+0000","last_peered":"2024-09-21T02:09:16.590616+0000","last_clean":"2024-09-21T02:09:16.590616+0000","last_became_active":"2024-09-21T02:09:11.546158+0000","last_became_peered":"2024-09-21T02:09:11.546158+0000","last_unstale":"2024-09-21T02:09:16.590616+0000","last_undegraded":"2024-09-21T02:09:16.590616+0000","last_fullsized":"2024-09-21T02:09:16.590616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588876+0000","last_change":"2024-09-21T02:09:15.612941+0000","last_active":"2024-09-21T02:09:16.588876+0000","last_peered":"2024-09-21T02:09:16.588876+0000","last_clean":"2024-09-21T02:09:16.588876+0000","last_became_active":"2024-09-21T02:09:15.612114+0000","last_became_peered":"2024-09-21T02:09:15.612114+0000","last_unstale":"2024-09-21T02:09:16.588876+0000","last_undegraded":"2024-09-21T02:09:16.588876+0000","last_fullsized":"2024-09-21T02:09:16.588876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590737+0000","last_change":"2024-09-21T02:09:10.523298+0000","last_active":"2024-09-21T02:09:16.590737+0000","last_peered":"2024-09-21T02:09:16.590737+0000","last_clean":"2024-09-21T02:09:16.590737+0000","last_became_active":"2024-09-21T02:09:10.521768+0000","last_became_peered":"2024-09-21T02:09:10.521768+0000","last_unstale":"2024-09-21T02:09:16.590737+0000","last_undegraded":"2024-09-21T02:09:16.590737+0000","last_fullsized":"2024-09-21T02:09:16.590737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723214+0000","last_change":"2024-09-21T02:09:11.573485+0000","last_active":"2024-09-21T02:09:16.723214+0000","last_peered":"2024-09-21T02:09:16.723214+0000","last_clean":"2024-09-21T02:09:16.723214+0000","last_became_active":"2024-09-21T02:09:11.572854+0000","last_became_peered":"2024-09-21T02:09:11.572854+0000","last_unstale":"2024-09-21T02:09:16.723214+0000","last_undegraded":"2024-09-21T02:09:16.723214+0000","last_fullsized":"2024-09-21T02:09:16.723214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591993+0000","last_change":"2024-09-21T02:09:15.629971+0000","last_active":"2024-09-21T02:09:16.591993+0000","last_peered":"2024-09-21T02:09:16.591993+0000","last_clean":"2024-09-21T02:09:16.591993+0000","last_became_active":"2024-09-21T02:09:15.628916+0000","last_became_peered":"2024-09-21T02:09:15.628916+0000","last_unstale":"2024-09-21T02:09:16.591993+0000","last_undegraded":"2024-09-21T02:09:16.591993+0000","last_fullsized":"2024-09-21T02:09:16.591993+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:16.541985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592632+0000","last_change":"2024-09-21T02:09:10.549868+0000","last_active":"2024-09-21T02:09:16.592632+0000","last_peered":"2024-09-21T02:09:16.592632+0000","last_clean":"2024-09-21T02:09:16.592632+0000","last_became_active":"2024-09-21T02:09:10.549154+0000","last_became_peered":"2024-09-21T02:09:10.549154+0000","last_unstale":"2024-09-21T02:09:16.592632+0000","last_undegraded":"2024-09-21T02:09:16.592632+0000","last_fullsized":"2024-09-21T02:09:16.592632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:42.470809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723996+0000","last_change":"2024-09-21T02:09:11.556559+0000","last_active":"2024-09-21T02:09:16.723996+0000","last_peered":"2024-09-21T02:09:16.723996+0000","last_clean":"2024-09-21T02:09:16.723996+0000","last_became_active":"2024-09-21T02:09:11.555720+0000","last_became_peered":"2024-09-21T02:09:11.555720+0000","last_unstale":"2024-09-21T02:09:16.723996+0000","last_undegraded":"2024-09-21T02:09:16.723996+0000","last_fullsized":"2024-09-21T02:09:16.723996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:43.740023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596031+0000","last_change":"2024-09-21T02:09:15.626432+0000","last_active":"2024-09-21T02:09:16.596031+0000","last_peered":"2024-09-21T02:09:16.596031+0000","last_clean":"2024-09-21T02:09:16.596031+0000","last_became_active":"2024-09-21T02:09:15.620746+0000","last_became_peered":"2024-09-21T02:09:15.620746+0000","last_unstale":"2024-09-21T02:09:16.596031+0000","last_undegraded":"2024-09-21T02:09:16.596031+0000","last_fullsized":"2024-09-21T02:09:16.596031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:46.776489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591909+0000","last_change":"2024-09-21T02:09:10.552140+0000","last_active":"2024-09-21T02:09:16.591909+0000","last_peered":"2024-09-21T02:09:16.591909+0000","last_clean":"2024-09-21T02:09:16.591909+0000","last_became_active":"2024-09-21T02:09:10.551607+0000","last_became_peered":"2024-09-21T02:09:10.551607+0000","last_unstale":"2024-09-21T02:09:16.591909+0000","last_undegraded":"2024-09-21T02:09:16.591909+0000","last_fullsized":"2024-09-21T02:09:16.591909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724056+0000","last_change":"2024-09-21T02:09:11.557870+0000","last_active":"2024-09-21T02:09:16.724056+0000","last_peered":"2024-09-21T02:09:16.724056+0000","last_clean":"2024-09-21T02:09:16.724056+0000","last_became_active":"2024-09-21T02:09:11.556258+0000","last_became_peered":"2024-09-21T02:09:11.556258+0000","last_unstale":"2024-09-21T02:09:16.724056+0000","last_undegraded":"2024-09-21T02:09:16.724056+0000","last_fullsized":"2024-09-21T02:09:16.724056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592325+0000","last_change":"2024-09-21T02:09:15.639499+0000","last_active":"2024-09-21T02:09:16.592325+0000","last_peered":"2024-09-21T02:09:16.592325+0000","last_clean":"2024-09-21T02:09:16.592325+0000","last_became_active":"2024-09-21T02:09:15.638499+0000","last_became_peered":"2024-09-21T02:09:15.638499+0000","last_unstale":"2024-09-21T02:09:16.592325+0000","last_undegraded":"2024-09-21T02:09:16.592325+0000","last_fullsized":"2024-09-21T02:09:16.592325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:22.032011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722630+0000","last_change":"2024-09-21T02:09:11.561488+0000","last_active":"2024-09-21T02:09:16.722630+0000","last_peered":"2024-09-21T02:09:16.722630+0000","last_clean":"2024-09-21T02:09:16.722630+0000","last_became_active":"2024-09-21T02:09:11.560683+0000","last_became_peered":"2024-09-21T02:09:11.560683+0000","last_unstale":"2024-09-21T02:09:16.722630+0000","last_undegraded":"2024-09-21T02:09:16.722630+0000","last_fullsized":"2024-09-21T02:09:16.722630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592616+0000","last_change":"2024-09-21T02:09:10.552547+0000","last_active":"2024-09-21T02:09:16.592616+0000","last_peered":"2024-09-21T02:09:16.592616+0000","last_clean":"2024-09-21T02:09:16.592616+0000","last_became_active":"2024-09-21T02:09:10.552056+0000","last_became_peered":"2024-09-21T02:09:10.552056+0000","last_unstale":"2024-09-21T02:09:16.592616+0000","last_undegraded":"2024-09-21T02:09:16.592616+0000","last_fullsized":"2024-09-21T02:09:16.592616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594025+0000","last_change":"2024-09-21T02:09:15.611360+0000","last_active":"2024-09-21T02:09:16.594025+0000","last_peered":"2024-09-21T02:09:16.594025+0000","last_clean":"2024-09-21T02:09:16.594025+0000","last_became_active":"2024-09-21T02:09:15.609991+0000","last_became_peered":"2024-09-21T02:09:15.609991+0000","last_unstale":"2024-09-21T02:09:16.594025+0000","last_undegraded":"2024-09-21T02:09:16.594025+0000","last_fullsized":"2024-09-21T02:09:16.594025+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:24.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589470+0000","last_change":"2024-09-21T02:09:11.545847+0000","last_active":"2024-09-21T02:09:16.589470+0000","last_peered":"2024-09-21T02:09:16.589470+0000","last_clean":"2024-09-21T02:09:16.589470+0000","last_became_active":"2024-09-21T02:09:11.543606+0000","last_became_peered":"2024-09-21T02:09:11.543606+0000","last_unstale":"2024-09-21T02:09:16.589470+0000","last_undegraded":"2024-09-21T02:09:16.589470+0000","last_fullsized":"2024-09-21T02:09:16.589470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720543+0000","last_change":"2024-09-21T02:09:10.524519+0000","last_active":"2024-09-21T02:09:16.720543+0000","last_peered":"2024-09-21T02:09:16.720543+0000","last_clean":"2024-09-21T02:09:16.720543+0000","last_became_active":"2024-09-21T02:09:10.524074+0000","last_became_peered":"2024-09-21T02:09:10.524074+0000","last_unstale":"2024-09-21T02:09:16.720543+0000","last_undegraded":"2024-09-21T02:09:16.720543+0000","last_fullsized":"2024-09-21T02:09:16.720543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594281+0000","last_change":"2024-09-21T02:09:15.618974+0000","last_active":"2024-09-21T02:09:16.594281+0000","last_peered":"2024-09-21T02:09:16.594281+0000","last_clean":"2024-09-21T02:09:16.594281+0000","last_became_active":"2024-09-21T02:09:15.617774+0000","last_became_peered":"2024-09-21T02:09:15.617774+0000","last_unstale":"2024-09-21T02:09:16.594281+0000","last_undegraded":"2024-09-21T02:09:16.594281+0000","last_fullsized":"2024-09-21T02:09:16.594281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:47.809682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589087+0000","last_change":"2024-09-21T02:09:11.538383+0000","last_active":"2024-09-21T02:09:16.589087+0000","last_peered":"2024-09-21T02:09:16.589087+0000","last_clean":"2024-09-21T02:09:16.589087+0000","last_became_active":"2024-09-21T02:09:11.537569+0000","last_became_peered":"2024-09-21T02:09:11.537569+0000","last_unstale":"2024-09-21T02:09:16.589087+0000","last_undegraded":"2024-09-21T02:09:16.589087+0000","last_fullsized":"2024-09-21T02:09:16.589087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720139+0000","last_change":"2024-09-21T02:09:10.537296+0000","last_active":"2024-09-21T02:09:16.720139+0000","last_peered":"2024-09-21T02:09:16.720139+0000","last_clean":"2024-09-21T02:09:16.720139+0000","last_became_active":"2024-09-21T02:09:10.536913+0000","last_became_peered":"2024-09-21T02:09:10.536913+0000","last_unstale":"2024-09-21T02:09:16.720139+0000","last_undegraded":"2024-09-21T02:09:16.720139+0000","last_fullsized":"2024-09-21T02:09:16.720139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720276+0000","last_change":"2024-09-21T02:09:15.606253+0000","last_active":"2024-09-21T02:09:16.720276+0000","last_peered":"2024-09-21T02:09:16.720276+0000","last_clean":"2024-09-21T02:09:16.720276+0000","last_became_active":"2024-09-21T02:09:15.605847+0000","last_became_peered":"2024-09-21T02:09:15.605847+0000","last_unstale":"2024-09-21T02:09:16.720276+0000","last_undegraded":"2024-09-21T02:09:16.720276+0000","last_fullsized":"2024-09-21T02:09:16.720276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594287+0000","last_change":"2024-09-21T02:09:11.541952+0000","last_active":"2024-09-21T02:09:16.594287+0000","last_peered":"2024-09-21T02:09:16.594287+0000","last_clean":"2024-09-21T02:09:16.594287+0000","last_became_active":"2024-09-21T02:09:11.540560+0000","last_became_peered":"2024-09-21T02:09:11.540560+0000","last_unstale":"2024-09-21T02:09:16.594287+0000","last_undegraded":"2024-09-21T02:09:16.594287+0000","last_fullsized":"2024-09-21T02:09:16.594287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:25.306228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591648+0000","last_change":"2024-09-21T02:09:10.544405+0000","last_active":"2024-09-21T02:09:16.591648+0000","last_peered":"2024-09-21T02:09:16.591648+0000","last_clean":"2024-09-21T02:09:16.591648+0000","last_became_active":"2024-09-21T02:09:10.540771+0000","last_became_peered":"2024-09-21T02:09:10.540771+0000","last_unstale":"2024-09-21T02:09:16.591648+0000","last_undegraded":"2024-09-21T02:09:16.591648+0000","last_fullsized":"2024-09-21T02:09:16.591648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593729+0000","last_change":"2024-09-21T02:09:15.626770+0000","last_active":"2024-09-21T02:09:16.593729+0000","last_peered":"2024-09-21T02:09:16.593729+0000","last_clean":"2024-09-21T02:09:16.593729+0000","last_became_active":"2024-09-21T02:09:15.623532+0000","last_became_peered":"2024-09-21T02:09:15.623532+0000","last_unstale":"2024-09-21T02:09:16.593729+0000","last_undegraded":"2024-09-21T02:09:16.593729+0000","last_fullsized":"2024-09-21T02:09:16.593729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:52.883467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722937+0000","last_change":"2024-09-21T02:09:11.539618+0000","last_active":"2024-09-21T02:09:16.722937+0000","last_peered":"2024-09-21T02:09:16.722937+0000","last_clean":"2024-09-21T02:09:16.722937+0000","last_became_active":"2024-09-21T02:09:11.538801+0000","last_became_peered":"2024-09-21T02:09:11.538801+0000","last_unstale":"2024-09-21T02:09:16.722937+0000","last_undegraded":"2024-09-21T02:09:16.722937+0000","last_fullsized":"2024-09-21T02:09:16.722937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726772+0000","last_change":"2024-09-21T02:09:10.545874+0000","last_active":"2024-09-21T02:09:16.726772+0000","last_peered":"2024-09-21T02:09:16.726772+0000","last_clean":"2024-09-21T02:09:16.726772+0000","last_became_active":"2024-09-21T02:09:10.545200+0000","last_became_peered":"2024-09-21T02:09:10.545200+0000","last_unstale":"2024-09-21T02:09:16.726772+0000","last_undegraded":"2024-09-21T02:09:16.726772+0000","last_fullsized":"2024-09-21T02:09:16.726772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593400+0000","last_change":"2024-09-21T02:09:15.630472+0000","last_active":"2024-09-21T02:09:16.593400+0000","last_peered":"2024-09-21T02:09:16.593400+0000","last_clean":"2024-09-21T02:09:16.593400+0000","last_became_active":"2024-09-21T02:09:15.628764+0000","last_became_peered":"2024-09-21T02:09:15.628764+0000","last_unstale":"2024-09-21T02:09:16.593400+0000","last_undegraded":"2024-09-21T02:09:16.593400+0000","last_fullsized":"2024-09-21T02:09:16.593400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:21.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591110+0000","last_change":"2024-09-21T02:09:11.543344+0000","last_active":"2024-09-21T02:09:16.591110+0000","last_peered":"2024-09-21T02:09:16.591110+0000","last_clean":"2024-09-21T02:09:16.591110+0000","last_became_active":"2024-09-21T02:09:11.542450+0000","last_became_peered":"2024-09-21T02:09:11.542450+0000","last_unstale":"2024-09-21T02:09:16.591110+0000","last_undegraded":"2024-09-21T02:09:16.591110+0000","last_fullsized":"2024-09-21T02:09:16.591110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:49.206820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719673+0000","last_change":"2024-09-21T02:09:10.541105+0000","last_active":"2024-09-21T02:09:16.719673+0000","last_peered":"2024-09-21T02:09:16.719673+0000","last_clean":"2024-09-21T02:09:16.719673+0000","last_became_active":"2024-09-21T02:09:10.540744+0000","last_became_peered":"2024-09-21T02:09:10.540744+0000","last_unstale":"2024-09-21T02:09:16.719673+0000","last_undegraded":"2024-09-21T02:09:16.719673+0000","last_fullsized":"2024-09-21T02:09:16.719673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:02.568801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588758+0000","last_change":"2024-09-21T02:09:15.613004+0000","last_active":"2024-09-21T02:09:16.588758+0000","last_peered":"2024-09-21T02:09:16.588758+0000","last_clean":"2024-09-21T02:09:16.588758+0000","last_became_active":"2024-09-21T02:09:15.612265+0000","last_became_peered":"2024-09-21T02:09:15.612265+0000","last_unstale":"2024-09-21T02:09:16.588758+0000","last_undegraded":"2024-09-21T02:09:16.588758+0000","last_fullsized":"2024-09-21T02:09:16.588758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721841+0000","last_change":"2024-09-21T02:09:11.528642+0000","last_active":"2024-09-21T02:09:16.721841+0000","last_peered":"2024-09-21T02:09:16.721841+0000","last_clean":"2024-09-21T02:09:16.721841+0000","last_became_active":"2024-09-21T02:09:11.526472+0000","last_became_peered":"2024-09-21T02:09:11.526472+0000","last_unstale":"2024-09-21T02:09:16.721841+0000","last_undegraded":"2024-09-21T02:09:16.721841+0000","last_fullsized":"2024-09-21T02:09:16.721841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593724+0000","last_change":"2024-09-21T02:09:10.553157+0000","last_active":"2024-09-21T02:09:16.593724+0000","last_peered":"2024-09-21T02:09:16.593724+0000","last_clean":"2024-09-21T02:09:16.593724+0000","last_became_active":"2024-09-21T02:09:10.552668+0000","last_became_peered":"2024-09-21T02:09:10.552668+0000","last_unstale":"2024-09-21T02:09:16.593724+0000","last_undegraded":"2024-09-21T02:09:16.593724+0000","last_fullsized":"2024-09-21T02:09:16.593724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:02.667450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723243+0000","last_change":"2024-09-21T02:09:15.626993+0000","last_active":"2024-09-21T02:09:16.723243+0000","last_peered":"2024-09-21T02:09:16.723243+0000","last_clean":"2024-09-21T02:09:16.723243+0000","last_became_active":"2024-09-21T02:09:15.626367+0000","last_became_peered":"2024-09-21T02:09:15.626367+0000","last_unstale":"2024-09-21T02:09:16.723243+0000","last_undegraded":"2024-09-21T02:09:16.723243+0000","last_fullsized":"2024-09-21T02:09:16.723243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:31.903642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724350+0000","last_change":"2024-09-21T02:09:11.557042+0000","last_active":"2024-09-21T02:09:16.724350+0000","last_peered":"2024-09-21T02:09:16.724350+0000","last_clean":"2024-09-21T02:09:16.724350+0000","last_became_active":"2024-09-21T02:09:11.555201+0000","last_became_peered":"2024-09-21T02:09:11.555201+0000","last_unstale":"2024-09-21T02:09:16.724350+0000","last_undegraded":"2024-09-21T02:09:16.724350+0000","last_fullsized":"2024-09-21T02:09:16.724350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722129+0000","last_change":"2024-09-21T02:09:10.516313+0000","last_active":"2024-09-21T02:09:16.722129+0000","last_peered":"2024-09-21T02:09:16.722129+0000","last_clean":"2024-09-21T02:09:16.722129+0000","last_became_active":"2024-09-21T02:09:10.515863+0000","last_became_peered":"2024-09-21T02:09:10.515863+0000","last_unstale":"2024-09-21T02:09:16.722129+0000","last_undegraded":"2024-09-21T02:09:16.722129+0000","last_fullsized":"2024-09-21T02:09:16.722129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719183+0000","last_change":"2024-09-21T02:09:15.624503+0000","last_active":"2024-09-21T02:09:16.719183+0000","last_peered":"2024-09-21T02:09:16.719183+0000","last_clean":"2024-09-21T02:09:16.719183+0000","last_became_active":"2024-09-21T02:09:15.622674+0000","last_became_peered":"2024-09-21T02:09:15.622674+0000","last_unstale":"2024-09-21T02:09:16.719183+0000","last_undegraded":"2024-09-21T02:09:16.719183+0000","last_fullsized":"2024-09-21T02:09:16.719183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:32.890032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596034+0000","last_change":"2024-09-21T02:09:11.550090+0000","last_active":"2024-09-21T02:09:16.596034+0000","last_peered":"2024-09-21T02:09:16.596034+0000","last_clean":"2024-09-21T02:09:16.596034+0000","last_became_active":"2024-09-21T02:09:11.549246+0000","last_became_peered":"2024-09-21T02:09:11.549246+0000","last_unstale":"2024-09-21T02:09:16.596034+0000","last_undegraded":"2024-09-21T02:09:16.596034+0000","last_fullsized":"2024-09-21T02:09:16.596034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.842699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725146+0000","last_change":"2024-09-21T02:09:10.545849+0000","last_active":"2024-09-21T02:09:16.725146+0000","last_peered":"2024-09-21T02:09:16.725146+0000","last_clean":"2024-09-21T02:09:16.725146+0000","last_became_active":"2024-09-21T02:09:10.544961+0000","last_became_peered":"2024-09-21T02:09:10.544961+0000","last_unstale":"2024-09-21T02:09:16.725146+0000","last_undegraded":"2024-09-21T02:09:16.725146+0000","last_fullsized":"2024-09-21T02:09:16.725146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:43.073819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721335+0000","last_change":"2024-09-21T02:09:15.611136+0000","last_active":"2024-09-21T02:09:16.721335+0000","last_peered":"2024-09-21T02:09:16.721335+0000","last_clean":"2024-09-21T02:09:16.721335+0000","last_became_active":"2024-09-21T02:09:15.610710+0000","last_became_peered":"2024-09-21T02:09:15.610710+0000","last_unstale":"2024-09-21T02:09:16.721335+0000","last_undegraded":"2024-09-21T02:09:16.721335+0000","last_fullsized":"2024-09-21T02:09:16.721335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:11.070735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719495+0000","last_change":"2024-09-21T02:09:11.526818+0000","last_active":"2024-09-21T02:09:16.719495+0000","last_peered":"2024-09-21T02:09:16.719495+0000","last_clean":"2024-09-21T02:09:16.719495+0000","last_became_active":"2024-09-21T02:09:11.525899+0000","last_became_peered":"2024-09-21T02:09:11.525899+0000","last_unstale":"2024-09-21T02:09:16.719495+0000","last_undegraded":"2024-09-21T02:09:16.719495+0000","last_fullsized":"2024-09-21T02:09:16.719495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720218+0000","last_change":"2024-09-21T02:09:10.526158+0000","last_active":"2024-09-21T02:09:16.720218+0000","last_peered":"2024-09-21T02:09:16.720218+0000","last_clean":"2024-09-21T02:09:16.720218+0000","last_became_active":"2024-09-21T02:09:10.525593+0000","last_became_peered":"2024-09-21T02:09:10.525593+0000","last_unstale":"2024-09-21T02:09:16.720218+0000","last_undegraded":"2024-09-21T02:09:16.720218+0000","last_fullsized":"2024-09-21T02:09:16.720218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720063+0000","last_change":"2024-09-21T02:09:15.623852+0000","last_active":"2024-09-21T02:09:16.720063+0000","last_peered":"2024-09-21T02:09:16.720063+0000","last_clean":"2024-09-21T02:09:16.720063+0000","last_became_active":"2024-09-21T02:09:15.623080+0000","last_became_peered":"2024-09-21T02:09:15.623080+0000","last_unstale":"2024-09-21T02:09:16.720063+0000","last_undegraded":"2024-09-21T02:09:16.720063+0000","last_fullsized":"2024-09-21T02:09:16.720063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:38.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722226+0000","last_change":"2024-09-21T02:09:11.558074+0000","last_active":"2024-09-21T02:09:16.722226+0000","last_peered":"2024-09-21T02:09:16.722226+0000","last_clean":"2024-09-21T02:09:16.722226+0000","last_became_active":"2024-09-21T02:09:11.557270+0000","last_became_peered":"2024-09-21T02:09:11.557270+0000","last_unstale":"2024-09-21T02:09:16.722226+0000","last_undegraded":"2024-09-21T02:09:16.722226+0000","last_fullsized":"2024-09-21T02:09:16.722226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:29.728033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590847+0000","last_change":"2024-09-21T02:09:10.540346+0000","last_active":"2024-09-21T02:09:16.590847+0000","last_peered":"2024-09-21T02:09:16.590847+0000","last_clean":"2024-09-21T02:09:16.590847+0000","last_became_active":"2024-09-21T02:09:10.539632+0000","last_became_peered":"2024-09-21T02:09:10.539632+0000","last_unstale":"2024-09-21T02:09:16.590847+0000","last_undegraded":"2024-09-21T02:09:16.590847+0000","last_fullsized":"2024-09-21T02:09:16.590847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722704+0000","last_change":"2024-09-21T02:09:15.636027+0000","last_active":"2024-09-21T02:09:16.722704+0000","last_peered":"2024-09-21T02:09:16.722704+0000","last_clean":"2024-09-21T02:09:16.722704+0000","last_became_active":"2024-09-21T02:09:15.635644+0000","last_became_peered":"2024-09-21T02:09:15.635644+0000","last_unstale":"2024-09-21T02:09:16.722704+0000","last_undegraded":"2024-09-21T02:09:16.722704+0000","last_fullsized":"2024-09-21T02:09:16.722704+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:46.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589275+0000","last_change":"2024-09-21T02:09:11.568395+0000","last_active":"2024-09-21T02:09:16.589275+0000","last_peered":"2024-09-21T02:09:16.589275+0000","last_clean":"2024-09-21T02:09:16.589275+0000","last_became_active":"2024-09-21T02:09:11.567951+0000","last_became_peered":"2024-09-21T02:09:11.567951+0000","last_unstale":"2024-09-21T02:09:16.589275+0000","last_undegraded":"2024-09-21T02:09:16.589275+0000","last_fullsized":"2024-09-21T02:09:16.589275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:38.502107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724904+0000","last_change":"2024-09-21T02:09:10.542137+0000","last_active":"2024-09-21T02:09:16.724904+0000","last_peered":"2024-09-21T02:09:16.724904+0000","last_clean":"2024-09-21T02:09:16.724904+0000","last_became_active":"2024-09-21T02:09:10.541324+0000","last_became_peered":"2024-09-21T02:09:10.541324+0000","last_unstale":"2024-09-21T02:09:16.724904+0000","last_undegraded":"2024-09-21T02:09:16.724904+0000","last_fullsized":"2024-09-21T02:09:16.724904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591215+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:09:16.591215+0000","last_peered":"2024-09-21T02:09:16.591215+0000","last_clean":"2024-09-21T02:09:16.591215+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:09:16.591215+0000","last_undegraded":"2024-09-21T02:09:16.591215+0000","last_fullsized":"2024-09-21T02:09:16.591215+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725602+0000","last_change":"2024-09-21T02:09:15.613320+0000","last_active":"2024-09-21T02:09:16.725602+0000","last_peered":"2024-09-21T02:09:16.725602+0000","last_clean":"2024-09-21T02:09:16.725602+0000","last_became_active":"2024-09-21T02:09:15.612050+0000","last_became_peered":"2024-09-21T02:09:15.612050+0000","last_unstale":"2024-09-21T02:09:16.725602+0000","last_undegraded":"2024-09-21T02:09:16.725602+0000","last_fullsized":"2024-09-21T02:09:16.725602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:54.519499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592039+0000","last_change":"2024-09-21T02:09:11.571228+0000","last_active":"2024-09-21T02:09:16.592039+0000","last_peered":"2024-09-21T02:09:16.592039+0000","last_clean":"2024-09-21T02:09:16.592039+0000","last_became_active":"2024-09-21T02:09:11.570627+0000","last_became_peered":"2024-09-21T02:09:11.570627+0000","last_unstale":"2024-09-21T02:09:16.592039+0000","last_undegraded":"2024-09-21T02:09:16.592039+0000","last_fullsized":"2024-09-21T02:09:16.592039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.513856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721976+0000","last_change":"2024-09-21T02:09:10.530153+0000","last_active":"2024-09-21T02:09:16.721976+0000","last_peered":"2024-09-21T02:09:16.721976+0000","last_clean":"2024-09-21T02:09:16.721976+0000","last_became_active":"2024-09-21T02:09:10.529650+0000","last_became_peered":"2024-09-21T02:09:10.529650+0000","last_unstale":"2024-09-21T02:09:16.721976+0000","last_undegraded":"2024-09-21T02:09:16.721976+0000","last_fullsized":"2024-09-21T02:09:16.721976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724037+0000","last_change":"2024-09-21T02:09:15.609642+0000","last_active":"2024-09-21T02:09:16.724037+0000","last_peered":"2024-09-21T02:09:16.724037+0000","last_clean":"2024-09-21T02:09:16.724037+0000","last_became_active":"2024-09-21T02:09:15.604449+0000","last_became_peered":"2024-09-21T02:09:15.604449+0000","last_unstale":"2024-09-21T02:09:16.724037+0000","last_undegraded":"2024-09-21T02:09:16.724037+0000","last_fullsized":"2024-09-21T02:09:16.724037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:55.894914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724291+0000","last_change":"2024-09-21T02:09:11.554539+0000","last_active":"2024-09-21T02:09:16.724291+0000","last_peered":"2024-09-21T02:09:16.724291+0000","last_clean":"2024-09-21T02:09:16.724291+0000","last_became_active":"2024-09-21T02:09:11.553712+0000","last_became_peered":"2024-09-21T02:09:11.553712+0000","last_unstale":"2024-09-21T02:09:16.724291+0000","last_undegraded":"2024-09-21T02:09:16.724291+0000","last_fullsized":"2024-09-21T02:09:16.724291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.229624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719880+0000","last_change":"2024-09-21T02:09:10.545134+0000","last_active":"2024-09-21T02:09:16.719880+0000","last_peered":"2024-09-21T02:09:16.719880+0000","last_clean":"2024-09-21T02:09:16.719880+0000","last_became_active":"2024-09-21T02:09:10.544792+0000","last_became_peered":"2024-09-21T02:09:10.544792+0000","last_unstale":"2024-09-21T02:09:16.719880+0000","last_undegraded":"2024-09-21T02:09:16.719880+0000","last_fullsized":"2024-09-21T02:09:16.719880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723145+0000","last_change":"2024-09-21T02:09:15.626483+0000","last_active":"2024-09-21T02:09:16.723145+0000","last_peered":"2024-09-21T02:09:16.723145+0000","last_clean":"2024-09-21T02:09:16.723145+0000","last_became_active":"2024-09-21T02:09:15.626132+0000","last_became_peered":"2024-09-21T02:09:15.626132+0000","last_unstale":"2024-09-21T02:09:16.723145+0000","last_undegraded":"2024-09-21T02:09:16.723145+0000","last_fullsized":"2024-09-21T02:09:16.723145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:04.011318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592725+0000","last_change":"2024-09-21T02:09:11.533777+0000","last_active":"2024-09-21T02:09:16.592725+0000","last_peered":"2024-09-21T02:09:16.592725+0000","last_clean":"2024-09-21T02:09:16.592725+0000","last_became_active":"2024-09-21T02:09:11.532928+0000","last_became_peered":"2024-09-21T02:09:11.532928+0000","last_unstale":"2024-09-21T02:09:16.592725+0000","last_undegraded":"2024-09-21T02:09:16.592725+0000","last_fullsized":"2024-09-21T02:09:16.592725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:21.176822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721695+0000","last_change":"2024-09-21T02:09:10.524899+0000","last_active":"2024-09-21T02:09:16.721695+0000","last_peered":"2024-09-21T02:09:16.721695+0000","last_clean":"2024-09-21T02:09:16.721695+0000","last_became_active":"2024-09-21T02:09:10.524447+0000","last_became_peered":"2024-09-21T02:09:10.524447+0000","last_unstale":"2024-09-21T02:09:16.721695+0000","last_undegraded":"2024-09-21T02:09:16.721695+0000","last_fullsized":"2024-09-21T02:09:16.721695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723182+0000","last_change":"2024-09-21T02:09:15.623410+0000","last_active":"2024-09-21T02:09:16.723182+0000","last_peered":"2024-09-21T02:09:16.723182+0000","last_clean":"2024-09-21T02:09:16.723182+0000","last_became_active":"2024-09-21T02:09:15.622194+0000","last_became_peered":"2024-09-21T02:09:15.622194+0000","last_unstale":"2024-09-21T02:09:16.723182+0000","last_undegraded":"2024-09-21T02:09:16.723182+0000","last_fullsized":"2024-09-21T02:09:16.723182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594437+0000","last_change":"2024-09-21T02:09:11.531628+0000","last_active":"2024-09-21T02:09:16.594437+0000","last_peered":"2024-09-21T02:09:16.594437+0000","last_clean":"2024-09-21T02:09:16.594437+0000","last_became_active":"2024-09-21T02:09:11.522487+0000","last_became_peered":"2024-09-21T02:09:11.522487+0000","last_unstale":"2024-09-21T02:09:16.594437+0000","last_undegraded":"2024-09-21T02:09:16.594437+0000","last_fullsized":"2024-09-21T02:09:16.594437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:31.403053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595653+0000","last_change":"2024-09-21T02:09:10.526000+0000","last_active":"2024-09-21T02:09:16.595653+0000","last_peered":"2024-09-21T02:09:16.595653+0000","last_clean":"2024-09-21T02:09:16.595653+0000","last_became_active":"2024-09-21T02:09:10.525267+0000","last_became_peered":"2024-09-21T02:09:10.525267+0000","last_unstale":"2024-09-21T02:09:16.595653+0000","last_undegraded":"2024-09-21T02:09:16.595653+0000","last_fullsized":"2024-09-21T02:09:16.595653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:32.448144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592641+0000","last_change":"2024-09-21T02:09:15.626495+0000","last_active":"2024-09-21T02:09:16.592641+0000","last_peered":"2024-09-21T02:09:16.592641+0000","last_clean":"2024-09-21T02:09:16.592641+0000","last_became_active":"2024-09-21T02:09:15.625151+0000","last_became_peered":"2024-09-21T02:09:15.625151+0000","last_unstale":"2024-09-21T02:09:16.592641+0000","last_undegraded":"2024-09-21T02:09:16.592641+0000","last_fullsized":"2024-09-21T02:09:16.592641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:23.340983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591341+0000","last_change":"2024-09-21T02:09:11.521241+0000","last_active":"2024-09-21T02:09:16.591341+0000","last_peered":"2024-09-21T02:09:16.591341+0000","last_clean":"2024-09-21T02:09:16.591341+0000","last_became_active":"2024-09-21T02:09:11.519883+0000","last_became_peered":"2024-09-21T02:09:11.519883+0000","last_unstale":"2024-09-21T02:09:16.591341+0000","last_undegraded":"2024-09-21T02:09:16.591341+0000","last_fullsized":"2024-09-21T02:09:16.591341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:26.260780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720026+0000","last_change":"2024-09-21T02:09:10.520638+0000","last_active":"2024-09-21T02:09:16.720026+0000","last_peered":"2024-09-21T02:09:16.720026+0000","last_clean":"2024-09-21T02:09:16.720026+0000","last_became_active":"2024-09-21T02:09:10.520131+0000","last_became_peered":"2024-09-21T02:09:10.520131+0000","last_unstale":"2024-09-21T02:09:16.720026+0000","last_undegraded":"2024-09-21T02:09:16.720026+0000","last_fullsized":"2024-09-21T02:09:16.720026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723034+0000","last_change":"2024-09-21T02:09:15.610523+0000","last_active":"2024-09-21T02:09:16.723034+0000","last_peered":"2024-09-21T02:09:16.723034+0000","last_clean":"2024-09-21T02:09:16.723034+0000","last_became_active":"2024-09-21T02:09:15.609441+0000","last_became_peered":"2024-09-21T02:09:15.609441+0000","last_unstale":"2024-09-21T02:09:16.723034+0000","last_undegraded":"2024-09-21T02:09:16.723034+0000","last_fullsized":"2024-09-21T02:09:16.723034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:36.011939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722864+0000","last_change":"2024-09-21T02:09:11.561816+0000","last_active":"2024-09-21T02:09:16.722864+0000","last_peered":"2024-09-21T02:09:16.722864+0000","last_clean":"2024-09-21T02:09:16.722864+0000","last_became_active":"2024-09-21T02:09:11.561231+0000","last_became_peered":"2024-09-21T02:09:11.561231+0000","last_unstale":"2024-09-21T02:09:16.722864+0000","last_undegraded":"2024-09-21T02:09:16.722864+0000","last_fullsized":"2024-09-21T02:09:16.722864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:38.366733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723173+0000","last_change":"2024-09-21T02:09:10.550508+0000","last_active":"2024-09-21T02:09:16.723173+0000","last_peered":"2024-09-21T02:09:16.723173+0000","last_clean":"2024-09-21T02:09:16.723173+0000","last_became_active":"2024-09-21T02:09:10.549913+0000","last_became_peered":"2024-09-21T02:09:10.549913+0000","last_unstale":"2024-09-21T02:09:16.723173+0000","last_undegraded":"2024-09-21T02:09:16.723173+0000","last_fullsized":"2024-09-21T02:09:16.723173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:15.201315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591175+0000","last_change":"2024-09-21T02:09:15.618146+0000","last_active":"2024-09-21T02:09:16.591175+0000","last_peered":"2024-09-21T02:09:16.591175+0000","last_clean":"2024-09-21T02:09:16.591175+0000","last_became_active":"2024-09-21T02:09:15.617195+0000","last_became_peered":"2024-09-21T02:09:15.617195+0000","last_unstale":"2024-09-21T02:09:16.591175+0000","last_undegraded":"2024-09-21T02:09:16.591175+0000","last_fullsized":"2024-09-21T02:09:16.591175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.452372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592327+0000","last_change":"2024-09-21T02:09:11.520992+0000","last_active":"2024-09-21T02:09:16.592327+0000","last_peered":"2024-09-21T02:09:16.592327+0000","last_clean":"2024-09-21T02:09:16.592327+0000","last_became_active":"2024-09-21T02:09:11.519892+0000","last_became_peered":"2024-09-21T02:09:11.519892+0000","last_unstale":"2024-09-21T02:09:16.592327+0000","last_undegraded":"2024-09-21T02:09:16.592327+0000","last_fullsized":"2024-09-21T02:09:16.592327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:34.673193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724869+0000","last_change":"2024-09-21T02:09:10.543820+0000","last_active":"2024-09-21T02:09:16.724869+0000","last_peered":"2024-09-21T02:09:16.724869+0000","last_clean":"2024-09-21T02:09:16.724869+0000","last_became_active":"2024-09-21T02:09:10.542944+0000","last_became_peered":"2024-09-21T02:09:10.542944+0000","last_unstale":"2024-09-21T02:09:16.724869+0000","last_undegraded":"2024-09-21T02:09:16.724869+0000","last_fullsized":"2024-09-21T02:09:16.724869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:59.409264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590058+0000","last_change":"2024-09-21T02:09:15.612341+0000","last_active":"2024-09-21T02:09:16.590058+0000","last_peered":"2024-09-21T02:09:16.590058+0000","last_clean":"2024-09-21T02:09:16.590058+0000","last_became_active":"2024-09-21T02:09:15.611864+0000","last_became_peered":"2024-09-21T02:09:15.611864+0000","last_unstale":"2024-09-21T02:09:16.590058+0000","last_undegraded":"2024-09-21T02:09:16.590058+0000","last_fullsized":"2024-09-21T02:09:16.590058+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:25.484143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723926+0000","last_change":"2024-09-21T02:09:11.557442+0000","last_active":"2024-09-21T02:09:16.723926+0000","last_peered":"2024-09-21T02:09:16.723926+0000","last_clean":"2024-09-21T02:09:16.723926+0000","last_became_active":"2024-09-21T02:09:11.557119+0000","last_became_peered":"2024-09-21T02:09:11.557119+0000","last_unstale":"2024-09-21T02:09:16.723926+0000","last_undegraded":"2024-09-21T02:09:16.723926+0000","last_fullsized":"2024-09-21T02:09:16.723926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723410+0000","last_change":"2024-09-21T02:09:10.530343+0000","last_active":"2024-09-21T02:09:16.723410+0000","last_peered":"2024-09-21T02:09:16.723410+0000","last_clean":"2024-09-21T02:09:16.723410+0000","last_became_active":"2024-09-21T02:09:10.529930+0000","last_became_peered":"2024-09-21T02:09:10.529930+0000","last_unstale":"2024-09-21T02:09:16.723410+0000","last_undegraded":"2024-09-21T02:09:16.723410+0000","last_fullsized":"2024-09-21T02:09:16.723410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725837+0000","last_change":"2024-09-21T02:09:15.635216+0000","last_active":"2024-09-21T02:09:16.725837+0000","last_peered":"2024-09-21T02:09:16.725837+0000","last_clean":"2024-09-21T02:09:16.725837+0000","last_became_active":"2024-09-21T02:09:15.626937+0000","last_became_peered":"2024-09-21T02:09:15.626937+0000","last_unstale":"2024-09-21T02:09:16.725837+0000","last_undegraded":"2024-09-21T02:09:16.725837+0000","last_fullsized":"2024-09-21T02:09:16.725837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591747+0000","last_change":"2024-09-21T02:09:11.538575+0000","last_active":"2024-09-21T02:09:16.591747+0000","last_peered":"2024-09-21T02:09:16.591747+0000","last_clean":"2024-09-21T02:09:16.591747+0000","last_became_active":"2024-09-21T02:09:11.535561+0000","last_became_peered":"2024-09-21T02:09:11.535561+0000","last_unstale":"2024-09-21T02:09:16.591747+0000","last_undegraded":"2024-09-21T02:09:16.591747+0000","last_fullsized":"2024-09-21T02:09:16.591747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:58.627736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:22.534320+0000","last_change":"2024-09-21T02:09:10.546712+0000","last_active":"2024-09-21T02:09:22.534320+0000","last_peered":"2024-09-21T02:09:22.534320+0000","last_clean":"2024-09-21T02:09:22.534320+0000","last_became_active":"2024-09-21T02:09:10.541968+0000","last_became_peered":"2024-09-21T02:09:10.541968+0000","last_unstale":"2024-09-21T02:09:22.534320+0000","last_undegraded":"2024-09-21T02:09:22.534320+0000","last_fullsized":"2024-09-21T02:09:22.534320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720210+0000","last_change":"2024-09-21T02:09:15.616116+0000","last_active":"2024-09-21T02:09:16.720210+0000","last_peered":"2024-09-21T02:09:16.720210+0000","last_clean":"2024-09-21T02:09:16.720210+0000","last_became_active":"2024-09-21T02:09:15.615337+0000","last_became_peered":"2024-09-21T02:09:15.615337+0000","last_unstale":"2024-09-21T02:09:16.720210+0000","last_undegraded":"2024-09-21T02:09:16.720210+0000","last_fullsized":"2024-09-21T02:09:16.720210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:35.472712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594788+0000","last_change":"2024-09-21T02:09:11.520271+0000","last_active":"2024-09-21T02:09:16.594788+0000","last_peered":"2024-09-21T02:09:16.594788+0000","last_clean":"2024-09-21T02:09:16.594788+0000","last_became_active":"2024-09-21T02:09:11.518967+0000","last_became_peered":"2024-09-21T02:09:11.518967+0000","last_unstale":"2024-09-21T02:09:16.594788+0000","last_undegraded":"2024-09-21T02:09:16.594788+0000","last_fullsized":"2024-09-21T02:09:16.594788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719861+0000","last_change":"2024-09-21T02:09:10.536568+0000","last_active":"2024-09-21T02:09:16.719861+0000","last_peered":"2024-09-21T02:09:16.719861+0000","last_clean":"2024-09-21T02:09:16.719861+0000","last_became_active":"2024-09-21T02:09:10.536266+0000","last_became_peered":"2024-09-21T02:09:10.536266+0000","last_unstale":"2024-09-21T02:09:16.719861+0000","last_undegraded":"2024-09-21T02:09:16.719861+0000","last_fullsized":"2024-09-21T02:09:16.719861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595233+0000","last_change":"2024-09-21T02:09:15.637262+0000","last_active":"2024-09-21T02:09:16.595233+0000","last_peered":"2024-09-21T02:09:16.595233+0000","last_clean":"2024-09-21T02:09:16.595233+0000","last_became_active":"2024-09-21T02:09:15.636305+0000","last_became_peered":"2024-09-21T02:09:15.636305+0000","last_unstale":"2024-09-21T02:09:16.595233+0000","last_undegraded":"2024-09-21T02:09:16.595233+0000","last_fullsized":"2024-09-21T02:09:16.595233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:36.447874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723352+0000","last_change":"2024-09-21T02:09:11.562979+0000","last_active":"2024-09-21T02:09:16.723352+0000","last_peered":"2024-09-21T02:09:16.723352+0000","last_clean":"2024-09-21T02:09:16.723352+0000","last_became_active":"2024-09-21T02:09:11.559009+0000","last_became_peered":"2024-09-21T02:09:11.559009+0000","last_unstale":"2024-09-21T02:09:16.723352+0000","last_undegraded":"2024-09-21T02:09:16.723352+0000","last_fullsized":"2024-09-21T02:09:16.723352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:27.296832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725417+0000","last_change":"2024-09-21T02:09:10.546521+0000","last_active":"2024-09-21T02:09:16.725417+0000","last_peered":"2024-09-21T02:09:16.725417+0000","last_clean":"2024-09-21T02:09:16.725417+0000","last_became_active":"2024-09-21T02:09:10.545864+0000","last_became_peered":"2024-09-21T02:09:10.545864+0000","last_unstale":"2024-09-21T02:09:16.725417+0000","last_undegraded":"2024-09-21T02:09:16.725417+0000","last_fullsized":"2024-09-21T02:09:16.725417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589335+0000","last_change":"2024-09-21T02:09:15.616179+0000","last_active":"2024-09-21T02:09:16.589335+0000","last_peered":"2024-09-21T02:09:16.589335+0000","last_clean":"2024-09-21T02:09:16.589335+0000","last_became_active":"2024-09-21T02:09:15.615344+0000","last_became_peered":"2024-09-21T02:09:15.615344+0000","last_unstale":"2024-09-21T02:09:16.589335+0000","last_undegraded":"2024-09-21T02:09:16.589335+0000","last_fullsized":"2024-09-21T02:09:16.589335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.519680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725123+0000","last_change":"2024-09-21T02:09:11.582832+0000","last_active":"2024-09-21T02:09:16.725123+0000","last_peered":"2024-09-21T02:09:16.725123+0000","last_clean":"2024-09-21T02:09:16.725123+0000","last_became_active":"2024-09-21T02:09:11.582121+0000","last_became_peered":"2024-09-21T02:09:11.582121+0000","last_unstale":"2024-09-21T02:09:16.725123+0000","last_undegraded":"2024-09-21T02:09:16.725123+0000","last_fullsized":"2024-09-21T02:09:16.725123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.829896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726463+0000","last_change":"2024-09-21T02:09:10.556671+0000","last_active":"2024-09-21T02:09:16.726463+0000","last_peered":"2024-09-21T02:09:16.726463+0000","last_clean":"2024-09-21T02:09:16.726463+0000","last_became_active":"2024-09-21T02:09:10.555875+0000","last_became_peered":"2024-09-21T02:09:10.555875+0000","last_unstale":"2024-09-21T02:09:16.726463+0000","last_undegraded":"2024-09-21T02:09:16.726463+0000","last_fullsized":"2024-09-21T02:09:16.726463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588965+0000","last_change":"2024-09-21T02:09:15.623197+0000","last_active":"2024-09-21T02:09:16.588965+0000","last_peered":"2024-09-21T02:09:16.588965+0000","last_clean":"2024-09-21T02:09:16.588965+0000","last_became_active":"2024-09-21T02:09:15.622340+0000","last_became_peered":"2024-09-21T02:09:15.622340+0000","last_unstale":"2024-09-21T02:09:16.588965+0000","last_undegraded":"2024-09-21T02:09:16.588965+0000","last_fullsized":"2024-09-21T02:09:16.588965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722347+0000","last_change":"2024-09-21T02:09:15.636227+0000","last_active":"2024-09-21T02:09:16.722347+0000","last_peered":"2024-09-21T02:09:16.722347+0000","last_clean":"2024-09-21T02:09:16.722347+0000","last_became_active":"2024-09-21T02:09:15.635814+0000","last_became_peered":"2024-09-21T02:09:15.635814+0000","last_unstale":"2024-09-21T02:09:16.722347+0000","last_undegraded":"2024-09-21T02:09:16.722347+0000","last_fullsized":"2024-09-21T02:09:16.722347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:37.103933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719830+0000","last_change":"2024-09-21T02:09:11.531143+0000","last_active":"2024-09-21T02:09:16.719830+0000","last_peered":"2024-09-21T02:09:16.719830+0000","last_clean":"2024-09-21T02:09:16.719830+0000","last_became_active":"2024-09-21T02:09:11.530348+0000","last_became_peered":"2024-09-21T02:09:11.530348+0000","last_unstale":"2024-09-21T02:09:16.719830+0000","last_undegraded":"2024-09-21T02:09:16.719830+0000","last_fullsized":"2024-09-21T02:09:16.719830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:21.823287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723931+0000","last_change":"2024-09-21T02:09:10.538525+0000","last_active":"2024-09-21T02:09:16.723931+0000","last_peered":"2024-09-21T02:09:16.723931+0000","last_clean":"2024-09-21T02:09:16.723931+0000","last_became_active":"2024-09-21T02:09:10.537466+0000","last_became_peered":"2024-09-21T02:09:10.537466+0000","last_unstale":"2024-09-21T02:09:16.723931+0000","last_undegraded":"2024-09-21T02:09:16.723931+0000","last_fullsized":"2024-09-21T02:09:16.723931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726815+0000","last_change":"2024-09-21T02:09:15.625976+0000","last_active":"2024-09-21T02:09:16.726815+0000","last_peered":"2024-09-21T02:09:16.726815+0000","last_clean":"2024-09-21T02:09:16.726815+0000","last_became_active":"2024-09-21T02:09:15.624692+0000","last_became_peered":"2024-09-21T02:09:15.624692+0000","last_unstale":"2024-09-21T02:09:16.726815+0000","last_undegraded":"2024-09-21T02:09:16.726815+0000","last_fullsized":"2024-09-21T02:09:16.726815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:18.311891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588890+0000","last_change":"2024-09-21T02:09:11.544871+0000","last_active":"2024-09-21T02:09:16.588890+0000","last_peered":"2024-09-21T02:09:16.588890+0000","last_clean":"2024-09-21T02:09:16.588890+0000","last_became_active":"2024-09-21T02:09:11.543403+0000","last_became_peered":"2024-09-21T02:09:11.543403+0000","last_unstale":"2024-09-21T02:09:16.588890+0000","last_undegraded":"2024-09-21T02:09:16.588890+0000","last_fullsized":"2024-09-21T02:09:16.588890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589183+0000","last_change":"2024-09-21T02:09:10.539555+0000","last_active":"2024-09-21T02:09:16.589183+0000","last_peered":"2024-09-21T02:09:16.589183+0000","last_clean":"2024-09-21T02:09:16.589183+0000","last_became_active":"2024-09-21T02:09:10.539163+0000","last_became_peered":"2024-09-21T02:09:10.539163+0000","last_unstale":"2024-09-21T02:09:16.589183+0000","last_undegraded":"2024-09-21T02:09:16.589183+0000","last_fullsized":"2024-09-21T02:09:16.589183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724509+0000","last_change":"2024-09-21T02:09:10.550879+0000","last_active":"2024-09-21T02:09:16.724509+0000","last_peered":"2024-09-21T02:09:16.724509+0000","last_clean":"2024-09-21T02:09:16.724509+0000","last_became_active":"2024-09-21T02:09:10.550147+0000","last_became_peered":"2024-09-21T02:09:10.550147+0000","last_unstale":"2024-09-21T02:09:16.724509+0000","last_undegraded":"2024-09-21T02:09:16.724509+0000","last_fullsized":"2024-09-21T02:09:16.724509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723322+0000","last_change":"2024-09-21T02:09:11.564520+0000","last_active":"2024-09-21T02:09:16.723322+0000","last_peered":"2024-09-21T02:09:16.723322+0000","last_clean":"2024-09-21T02:09:16.723322+0000","last_became_active":"2024-09-21T02:09:11.563870+0000","last_became_peered":"2024-09-21T02:09:11.563870+0000","last_unstale":"2024-09-21T02:09:16.723322+0000","last_undegraded":"2024-09-21T02:09:16.723322+0000","last_fullsized":"2024-09-21T02:09:16.723322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:20.175775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721646+0000","last_change":"2024-09-21T02:09:15.623267+0000","last_active":"2024-09-21T02:09:16.721646+0000","last_peered":"2024-09-21T02:09:16.721646+0000","last_clean":"2024-09-21T02:09:16.721646+0000","last_became_active":"2024-09-21T02:09:15.622816+0000","last_became_peered":"2024-09-21T02:09:15.622816+0000","last_unstale":"2024-09-21T02:09:16.721646+0000","last_undegraded":"2024-09-21T02:09:16.721646+0000","last_fullsized":"2024-09-21T02:09:16.721646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:11.398708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589624+0000","last_change":"2024-09-21T02:09:10.545988+0000","last_active":"2024-09-21T02:09:16.589624+0000","last_peered":"2024-09-21T02:09:16.589624+0000","last_clean":"2024-09-21T02:09:16.589624+0000","last_became_active":"2024-09-21T02:09:10.545537+0000","last_became_peered":"2024-09-21T02:09:10.545537+0000","last_unstale":"2024-09-21T02:09:16.589624+0000","last_undegraded":"2024-09-21T02:09:16.589624+0000","last_fullsized":"2024-09-21T02:09:16.589624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592924+0000","last_change":"2024-09-21T02:09:11.527921+0000","last_active":"2024-09-21T02:09:16.592924+0000","last_peered":"2024-09-21T02:09:16.592924+0000","last_clean":"2024-09-21T02:09:16.592924+0000","last_became_active":"2024-09-21T02:09:11.527033+0000","last_became_peered":"2024-09-21T02:09:11.527033+0000","last_unstale":"2024-09-21T02:09:16.592924+0000","last_undegraded":"2024-09-21T02:09:16.592924+0000","last_fullsized":"2024-09-21T02:09:16.592924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590282+0000","last_change":"2024-09-21T02:09:15.608837+0000","last_active":"2024-09-21T02:09:16.590282+0000","last_peered":"2024-09-21T02:09:16.590282+0000","last_clean":"2024-09-21T02:09:16.590282+0000","last_became_active":"2024-09-21T02:09:15.607905+0000","last_became_peered":"2024-09-21T02:09:15.607905+0000","last_unstale":"2024-09-21T02:09:16.590282+0000","last_undegraded":"2024-09-21T02:09:16.590282+0000","last_fullsized":"2024-09-21T02:09:16.590282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:15.347636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724603+0000","last_change":"2024-09-21T02:09:11.562756+0000","last_active":"2024-09-21T02:09:16.724603+0000","last_peered":"2024-09-21T02:09:16.724603+0000","last_clean":"2024-09-21T02:09:16.724603+0000","last_became_active":"2024-09-21T02:09:11.560604+0000","last_became_peered":"2024-09-21T02:09:11.560604+0000","last_unstale":"2024-09-21T02:09:16.724603+0000","last_undegraded":"2024-09-21T02:09:16.724603+0000","last_fullsized":"2024-09-21T02:09:16.724603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719949+0000","last_change":"2024-09-21T02:09:11.566770+0000","last_active":"2024-09-21T02:09:16.719949+0000","last_peered":"2024-09-21T02:09:16.719949+0000","last_clean":"2024-09-21T02:09:16.719949+0000","last_became_active":"2024-09-21T02:09:11.566425+0000","last_became_peered":"2024-09-21T02:09:11.566425+0000","last_unstale":"2024-09-21T02:09:16.719949+0000","last_undegraded":"2024-09-21T02:09:16.719949+0000","last_fullsized":"2024-09-21T02:09:16.719949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724847+0000","last_change":"2024-09-21T02:09:11.572089+0000","last_active":"2024-09-21T02:09:16.724847+0000","last_peered":"2024-09-21T02:09:16.724847+0000","last_clean":"2024-09-21T02:09:16.724847+0000","last_became_active":"2024-09-21T02:09:11.566576+0000","last_became_peered":"2024-09-21T02:09:11.566576+0000","last_unstale":"2024-09-21T02:09:16.724847+0000","last_undegraded":"2024-09-21T02:09:16.724847+0000","last_fullsized":"2024-09-21T02:09:16.724847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:42.706829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723234+0000","last_change":"2024-09-21T02:09:11.564192+0000","last_active":"2024-09-21T02:09:16.723234+0000","last_peered":"2024-09-21T02:09:16.723234+0000","last_clean":"2024-09-21T02:09:16.723234+0000","last_became_active":"2024-09-21T02:09:11.559079+0000","last_became_peered":"2024-09-21T02:09:11.559079+0000","last_unstale":"2024-09-21T02:09:16.723234+0000","last_undegraded":"2024-09-21T02:09:16.723234+0000","last_fullsized":"2024-09-21T02:09:16.723234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590380+0000","last_change":"2024-09-21T02:09:11.540588+0000","last_active":"2024-09-21T02:09:16.590380+0000","last_peered":"2024-09-21T02:09:16.590380+0000","last_clean":"2024-09-21T02:09:16.590380+0000","last_became_active":"2024-09-21T02:09:11.539779+0000","last_became_peered":"2024-09-21T02:09:11.539779+0000","last_unstale":"2024-09-21T02:09:16.590380+0000","last_undegraded":"2024-09-21T02:09:16.590380+0000","last_fullsized":"2024-09-21T02:09:16.590380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:24.826345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719412+0000","last_change":"2024-09-21T02:09:11.530898+0000","last_active":"2024-09-21T02:09:16.719412+0000","last_peered":"2024-09-21T02:09:16.719412+0000","last_clean":"2024-09-21T02:09:16.719412+0000","last_became_active":"2024-09-21T02:09:11.530100+0000","last_became_peered":"2024-09-21T02:09:11.530100+0000","last_unstale":"2024-09-21T02:09:16.719412+0000","last_undegraded":"2024-09-21T02:09:16.719412+0000","last_fullsized":"2024-09-21T02:09:16.719412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726838+0000","last_change":"2024-09-21T02:09:11.549198+0000","last_active":"2024-09-21T02:09:16.726838+0000","last_peered":"2024-09-21T02:09:16.726838+0000","last_clean":"2024-09-21T02:09:16.726838+0000","last_became_active":"2024-09-21T02:09:11.548421+0000","last_became_peered":"2024-09-21T02:09:11.548421+0000","last_unstale":"2024-09-21T02:09:16.726838+0000","last_undegraded":"2024-09-21T02:09:16.726838+0000","last_fullsized":"2024-09-21T02:09:16.726838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:22.505823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590900+0000","last_change":"2024-09-21T02:09:11.552426+0000","last_active":"2024-09-21T02:09:16.590900+0000","last_peered":"2024-09-21T02:09:16.590900+0000","last_clean":"2024-09-21T02:09:16.590900+0000","last_became_active":"2024-09-21T02:09:11.551844+0000","last_became_peered":"2024-09-21T02:09:11.551844+0000","last_unstale":"2024-09-21T02:09:16.590900+0000","last_undegraded":"2024-09-21T02:09:16.590900+0000","last_fullsized":"2024-09-21T02:09:16.590900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723635+0000","last_change":"2024-09-21T02:09:11.566549+0000","last_active":"2024-09-21T02:09:16.723635+0000","last_peered":"2024-09-21T02:09:16.723635+0000","last_clean":"2024-09-21T02:09:16.723635+0000","last_became_active":"2024-09-21T02:09:11.565905+0000","last_became_peered":"2024-09-21T02:09:11.565905+0000","last_unstale":"2024-09-21T02:09:16.723635+0000","last_undegraded":"2024-09-21T02:09:16.723635+0000","last_fullsized":"2024-09-21T02:09:16.723635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.460538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589185+0000","last_change":"2024-09-21T02:09:11.552230+0000","last_active":"2024-09-21T02:09:16.589185+0000","last_peered":"2024-09-21T02:09:16.589185+0000","last_clean":"2024-09-21T02:09:16.589185+0000","last_became_active":"2024-09-21T02:09:11.551873+0000","last_became_peered":"2024-09-21T02:09:11.551873+0000","last_unstale":"2024-09-21T02:09:16.589185+0000","last_undegraded":"2024-09-21T02:09:16.589185+0000","last_fullsized":"2024-09-21T02:09:16.589185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724470+0000","last_change":"2024-09-21T02:09:11.581809+0000","last_active":"2024-09-21T02:09:16.724470+0000","last_peered":"2024-09-21T02:09:16.724470+0000","last_clean":"2024-09-21T02:09:16.724470+0000","last_became_active":"2024-09-21T02:09:11.581010+0000","last_became_peered":"2024-09-21T02:09:11.581010+0000","last_unstale":"2024-09-21T02:09:16.724470+0000","last_undegraded":"2024-09-21T02:09:16.724470+0000","last_fullsized":"2024-09-21T02:09:16.724470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:19.271324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722807+0000","last_change":"2024-09-21T02:09:11.558586+0000","last_active":"2024-09-21T02:09:16.722807+0000","last_peered":"2024-09-21T02:09:16.722807+0000","last_clean":"2024-09-21T02:09:16.722807+0000","last_became_active":"2024-09-21T02:09:11.558126+0000","last_became_peered":"2024-09-21T02:09:11.558126+0000","last_unstale":"2024-09-21T02:09:16.722807+0000","last_undegraded":"2024-09-21T02:09:16.722807+0000","last_fullsized":"2024-09-21T02:09:16.722807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:10.080231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725755+0000","last_change":"2024-09-21T02:09:11.561183+0000","last_active":"2024-09-21T02:09:16.725755+0000","last_peered":"2024-09-21T02:09:16.725755+0000","last_clean":"2024-09-21T02:09:16.725755+0000","last_became_active":"2024-09-21T02:09:11.560260+0000","last_became_peered":"2024-09-21T02:09:11.560260+0000","last_unstale":"2024-09-21T02:09:16.725755+0000","last_undegraded":"2024-09-21T02:09:16.725755+0000","last_fullsized":"2024-09-21T02:09:16.725755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593415+0000","last_change":"2024-09-21T02:09:11.540272+0000","last_active":"2024-09-21T02:09:16.593415+0000","last_peered":"2024-09-21T02:09:16.593415+0000","last_clean":"2024-09-21T02:09:16.593415+0000","last_became_active":"2024-09-21T02:09:11.539409+0000","last_became_peered":"2024-09-21T02:09:11.539409+0000","last_unstale":"2024-09-21T02:09:16.593415+0000","last_undegraded":"2024-09-21T02:09:16.593415+0000","last_fullsized":"2024-09-21T02:09:16.593415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:28.820372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721399+0000","last_change":"2024-09-21T02:09:11.544995+0000","last_active":"2024-09-21T02:09:16.721399+0000","last_peered":"2024-09-21T02:09:16.721399+0000","last_clean":"2024-09-21T02:09:16.721399+0000","last_became_active":"2024-09-21T02:09:11.544306+0000","last_became_peered":"2024-09-21T02:09:11.544306+0000","last_unstale":"2024-09-21T02:09:16.721399+0000","last_undegraded":"2024-09-21T02:09:16.721399+0000","last_fullsized":"2024-09-21T02:09:16.721399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725507+0000","last_change":"2024-09-21T02:09:11.582210+0000","last_active":"2024-09-21T02:09:16.725507+0000","last_peered":"2024-09-21T02:09:16.725507+0000","last_clean":"2024-09-21T02:09:16.725507+0000","last_became_active":"2024-09-21T02:09:11.581734+0000","last_became_peered":"2024-09-21T02:09:11.581734+0000","last_unstale":"2024-09-21T02:09:16.725507+0000","last_undegraded":"2024-09-21T02:09:16.725507+0000","last_fullsized":"2024-09-21T02:09:16.725507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:01.926437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723576+0000","last_change":"2024-09-21T02:09:11.574300+0000","last_active":"2024-09-21T02:09:16.723576+0000","last_peered":"2024-09-21T02:09:16.723576+0000","last_clean":"2024-09-21T02:09:16.723576+0000","last_became_active":"2024-09-21T02:09:11.573443+0000","last_became_peered":"2024-09-21T02:09:11.573443+0000","last_unstale":"2024-09-21T02:09:16.723576+0000","last_undegraded":"2024-09-21T02:09:16.723576+0000","last_fullsized":"2024-09-21T02:09:16.723576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720120+0000","last_change":"2024-09-21T02:09:11.518238+0000","last_active":"2024-09-21T02:09:16.720120+0000","last_peered":"2024-09-21T02:09:16.720120+0000","last_clean":"2024-09-21T02:09:16.720120+0000","last_became_active":"2024-09-21T02:09:11.517379+0000","last_became_peered":"2024-09-21T02:09:11.517379+0000","last_unstale":"2024-09-21T02:09:16.720120+0000","last_undegraded":"2024-09-21T02:09:16.720120+0000","last_fullsized":"2024-09-21T02:09:16.720120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723824+0000","last_change":"2024-09-21T02:09:11.536912+0000","last_active":"2024-09-21T02:09:16.723824+0000","last_peered":"2024-09-21T02:09:16.723824+0000","last_clean":"2024-09-21T02:09:16.723824+0000","last_became_active":"2024-09-21T02:09:11.535607+0000","last_became_peered":"2024-09-21T02:09:11.535607+0000","last_unstale":"2024-09-21T02:09:16.723824+0000","last_undegraded":"2024-09-21T02:09:16.723824+0000","last_fullsized":"2024-09-21T02:09:16.723824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589439+0000","last_change":"2024-09-21T02:09:11.546912+0000","last_active":"2024-09-21T02:09:16.589439+0000","last_peered":"2024-09-21T02:09:16.589439+0000","last_clean":"2024-09-21T02:09:16.589439+0000","last_became_active":"2024-09-21T02:09:11.543438+0000","last_became_peered":"2024-09-21T02:09:11.543438+0000","last_unstale":"2024-09-21T02:09:16.589439+0000","last_undegraded":"2024-09-21T02:09:16.589439+0000","last_fullsized":"2024-09-21T02:09:16.589439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595165+0000","last_change":"2024-09-21T02:09:11.547068+0000","last_active":"2024-09-21T02:09:16.595165+0000","last_peered":"2024-09-21T02:09:16.595165+0000","last_clean":"2024-09-21T02:09:16.595165+0000","last_became_active":"2024-09-21T02:09:11.545661+0000","last_became_peered":"2024-09-21T02:09:11.545661+0000","last_unstale":"2024-09-21T02:09:16.595165+0000","last_undegraded":"2024-09-21T02:09:16.595165+0000","last_fullsized":"2024-09-21T02:09:16.595165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:00.421056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725978+0000","last_change":"2024-09-21T02:09:11.571193+0000","last_active":"2024-09-21T02:09:16.725978+0000","last_peered":"2024-09-21T02:09:16.725978+0000","last_clean":"2024-09-21T02:09:16.725978+0000","last_became_active":"2024-09-21T02:09:11.563913+0000","last_became_peered":"2024-09-21T02:09:11.563913+0000","last_unstale":"2024-09-21T02:09:16.725978+0000","last_undegraded":"2024-09-21T02:09:16.725978+0000","last_fullsized":"2024-09-21T02:09:16.725978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726661+0000","last_change":"2024-09-21T02:09:11.564018+0000","last_active":"2024-09-21T02:09:16.726661+0000","last_peered":"2024-09-21T02:09:16.726661+0000","last_clean":"2024-09-21T02:09:16.726661+0000","last_became_active":"2024-09-21T02:09:11.563374+0000","last_became_peered":"2024-09-21T02:09:11.563374+0000","last_unstale":"2024-09-21T02:09:16.726661+0000","last_undegraded":"2024-09-21T02:09:16.726661+0000","last_fullsized":"2024-09-21T02:09:16.726661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592542+0000","last_change":"2024-09-21T02:09:11.571561+0000","last_active":"2024-09-21T02:09:16.592542+0000","last_peered":"2024-09-21T02:09:16.592542+0000","last_clean":"2024-09-21T02:09:16.592542+0000","last_became_active":"2024-09-21T02:09:11.570942+0000","last_became_peered":"2024-09-21T02:09:11.570942+0000","last_unstale":"2024-09-21T02:09:16.592542+0000","last_undegraded":"2024-09-21T02:09:16.592542+0000","last_fullsized":"2024-09-21T02:09:16.592542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:07.001076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588463+0000","last_change":"2024-09-21T02:09:11.568051+0000","last_active":"2024-09-21T02:09:16.588463+0000","last_peered":"2024-09-21T02:09:16.588463+0000","last_clean":"2024-09-21T02:09:16.588463+0000","last_became_active":"2024-09-21T02:09:11.567478+0000","last_became_peered":"2024-09-21T02:09:11.567478+0000","last_unstale":"2024-09-21T02:09:16.588463+0000","last_undegraded":"2024-09-21T02:09:16.588463+0000","last_fullsized":"2024-09-21T02:09:16.588463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591178+0000","last_change":"2024-09-21T02:09:11.536755+0000","last_active":"2024-09-21T02:09:16.591178+0000","last_peered":"2024-09-21T02:09:16.591178+0000","last_clean":"2024-09-21T02:09:16.591178+0000","last_became_active":"2024-09-21T02:09:11.535871+0000","last_became_peered":"2024-09-21T02:09:11.535871+0000","last_unstale":"2024-09-21T02:09:16.591178+0000","last_undegraded":"2024-09-21T02:09:16.591178+0000","last_fullsized":"2024-09-21T02:09:16.591178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:00.938047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724464+0000","last_change":"2024-09-21T02:09:11.572153+0000","last_active":"2024-09-21T02:09:16.724464+0000","last_peered":"2024-09-21T02:09:16.724464+0000","last_clean":"2024-09-21T02:09:16.724464+0000","last_became_active":"2024-09-21T02:09:11.567470+0000","last_became_peered":"2024-09-21T02:09:11.567470+0000","last_unstale":"2024-09-21T02:09:16.724464+0000","last_undegraded":"2024-09-21T02:09:16.724464+0000","last_fullsized":"2024-09-21T02:09:16.724464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:24.775708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591826+0000","last_change":"2024-09-21T02:09:11.532323+0000","last_active":"2024-09-21T02:09:16.591826+0000","last_peered":"2024-09-21T02:09:16.591826+0000","last_clean":"2024-09-21T02:09:16.591826+0000","last_became_active":"2024-09-21T02:09:11.531480+0000","last_became_peered":"2024-09-21T02:09:11.531480+0000","last_unstale":"2024-09-21T02:09:16.591826+0000","last_undegraded":"2024-09-21T02:09:16.591826+0000","last_fullsized":"2024-09-21T02:09:16.591826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:40.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723696+0000","last_change":"2024-09-21T02:09:11.575204+0000","last_active":"2024-09-21T02:09:16.723696+0000","last_peered":"2024-09-21T02:09:16.723696+0000","last_clean":"2024-09-21T02:09:16.723696+0000","last_became_active":"2024-09-21T02:09:11.567379+0000","last_became_peered":"2024-09-21T02:09:11.567379+0000","last_unstale":"2024-09-21T02:09:16.723696+0000","last_undegraded":"2024-09-21T02:09:16.723696+0000","last_fullsized":"2024-09-21T02:09:16.723696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:48.813808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722559+0000","last_change":"2024-09-21T02:09:11.540599+0000","last_active":"2024-09-21T02:09:16.722559+0000","last_peered":"2024-09-21T02:09:16.722559+0000","last_clean":"2024-09-21T02:09:16.722559+0000","last_became_active":"2024-09-21T02:09:11.539829+0000","last_became_peered":"2024-09-21T02:09:11.539829+0000","last_unstale":"2024-09-21T02:09:16.722559+0000","last_undegraded":"2024-09-21T02:09:16.722559+0000","last_fullsized":"2024-09-21T02:09:16.722559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:54.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719555+0000","last_change":"2024-09-21T02:09:11.534649+0000","last_active":"2024-09-21T02:09:16.719555+0000","last_peered":"2024-09-21T02:09:16.719555+0000","last_clean":"2024-09-21T02:09:16.719555+0000","last_became_active":"2024-09-21T02:09:11.533392+0000","last_became_peered":"2024-09-21T02:09:11.533392+0000","last_unstale":"2024-09-21T02:09:16.719555+0000","last_undegraded":"2024-09-21T02:09:16.719555+0000","last_fullsized":"2024-09-21T02:09:16.719555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592021+0000","last_change":"2024-09-21T02:09:11.546630+0000","last_active":"2024-09-21T02:09:16.592021+0000","last_peered":"2024-09-21T02:09:16.592021+0000","last_clean":"2024-09-21T02:09:16.592021+0000","last_became_active":"2024-09-21T02:09:11.545498+0000","last_became_peered":"2024-09-21T02:09:11.545498+0000","last_unstale":"2024-09-21T02:09:16.592021+0000","last_undegraded":"2024-09-21T02:09:16.592021+0000","last_fullsized":"2024-09-21T02:09:16.592021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:25.840553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":699032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":108172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":108082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":282624,"data_stored":108242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":699088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:09:38.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:09:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 -- 172.21.15.96:0/859970470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415fb60 msgr2=0x7febd40d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 --2- 172.21.15.96:0/859970470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415fb60 0x7febd40d0460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7febd40737e0 tx=0x7febc401c890 comp rx=0 tx=0).stop 2024-09-21T02:09:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 -- 172.21.15.96:0/859970470 shutdown_connections 2024-09-21T02:09:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 --2- 172.21.15.96:0/859970470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415fb60 0x7febd40d0460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 --2- 172.21.15.96:0/859970470 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7febd415f750 0x7febd40cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 --2- 172.21.15.96:0/859970470 >> v2:172.21.15.122:3300/0 conn(0x7febd415b310 0x7febd40cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 -- 172.21.15.96:0/859970470 >> 172.21.15.96:0/859970470 conn(0x7febd406fbe0 msgr2=0x7febd406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:38.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 -- 172.21.15.96:0/859970470 shutdown_connections 2024-09-21T02:09:38.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 -- 172.21.15.96:0/859970470 wait complete. 2024-09-21T02:09:38.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.832+0000 7febd9aa7640 1 Processor -- start 2024-09-21T02:09:38.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- start start 2024-09-21T02:09:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7febd415b310 0x7febd41fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 0x7febd41f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febd415fb60 0x7febd41462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7febd406e3b0 con 0x7febd415f750 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7febd406e230 con 0x7febd415fb60 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7febd406e530 con 0x7febd415b310 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 0x7febd41f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 0x7febd41f8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42708/0 (socket says 172.21.15.96:42708) 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 -- 172.21.15.96:0/1135873314 learned_addr learned my addr 172.21.15.96:0/1135873314 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd37fe640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7febd415b310 0x7febd41fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd3fff640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febd415fb60 0x7febd41462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 -- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7febd415b310 msgr2=0x7febd41fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7febd415b310 0x7febd41fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 -- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febd415fb60 msgr2=0x7febd41462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febd415fb60 0x7febd41462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 -- 172.21.15.96:0/1135873314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd41fc950 con 0x7febd415f750 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd3fff640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febd415fb60 0x7febd41462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd2ffd640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 0x7febd41f8de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c940 tx=0x7febbc00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc004220 con 0x7febd415f750 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febbc0079b0 con 0x7febd415f750 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd41f8620 con 0x7febd415f750 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc0044b0 con 0x7febd415f750 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd415c560 con 0x7febd415f750 2024-09-21T02:09:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd41f8410 con 0x7febd415f750 2024-09-21T02:09:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.836+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7febbc003600 con 0x7febd415f750 2024-09-21T02:09:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.840+0000 7febcbfff640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7febb4073200 0x7febb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.840+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7febbc0947e0 con 0x7febd415f750 2024-09-21T02:09:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.840+0000 7febd37fe640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7febb4073200 0x7febb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.840+0000 7febd37fe640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7febb4073200 0x7febb4075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febcc005cd0 tx=0x7febcc00a760 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:38.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:38.844+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7febd41fc950 con 0x7febd415f750 2024-09-21T02:09:39.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.000+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febd415aab0 con 0x7febb4073200 2024-09-21T02:09:39.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.016+0000 7febcbfff640 1 -- 172.21.15.96:0/1135873314 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332520 (secure 0 0 0) 0x7febd415aab0 con 0x7febb4073200 2024-09-21T02:09:39.023 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:09:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7febb4073200 msgr2=0x7febb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7febb4073200 0x7febb4075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febcc005cd0 tx=0x7febcc00a760 comp rx=0 tx=0).stop 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 msgr2=0x7febd41f8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 0x7febd41f8de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c940 tx=0x7febbc00ce00 comp rx=0 tx=0).stop 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 shutdown_connections 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7febb4073200 0x7febb4075710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febd415fb60 0x7febd41462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd415f750 0x7febd41f8de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 --2- 172.21.15.96:0/1135873314 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7febd415b310 0x7febd41fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 >> 172.21.15.96:0/1135873314 conn(0x7febd406fbe0 msgr2=0x7febd40b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 shutdown_connections 2024-09-21T02:09:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.020+0000 7febd9aa7640 1 -- 172.21.15.96:0/1135873314 wait complete. 2024-09-21T02:09:39.042 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-21T02:09:38.507684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223768,"kb_used_data":3368,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750952,"statfs":{"total":773094113280,"available":772864974848,"internally_reserved":0,"allocated":3448832,"data_stored":2054672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005904"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723570+0000","last_change":"2024-09-21T02:09:10.527965+0000","last_active":"2024-09-21T02:09:16.723570+0000","last_peered":"2024-09-21T02:09:16.723570+0000","last_clean":"2024-09-21T02:09:16.723570+0000","last_became_active":"2024-09-21T02:09:10.527508+0000","last_became_peered":"2024-09-21T02:09:10.527508+0000","last_unstale":"2024-09-21T02:09:16.723570+0000","last_undegraded":"2024-09-21T02:09:16.723570+0000","last_fullsized":"2024-09-21T02:09:16.723570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:35.986615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590616+0000","last_change":"2024-09-21T02:09:11.546557+0000","last_active":"2024-09-21T02:09:16.590616+0000","last_peered":"2024-09-21T02:09:16.590616+0000","last_clean":"2024-09-21T02:09:16.590616+0000","last_became_active":"2024-09-21T02:09:11.546158+0000","last_became_peered":"2024-09-21T02:09:11.546158+0000","last_unstale":"2024-09-21T02:09:16.590616+0000","last_undegraded":"2024-09-21T02:09:16.590616+0000","last_fullsized":"2024-09-21T02:09:16.590616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588876+0000","last_change":"2024-09-21T02:09:15.612941+0000","last_active":"2024-09-21T02:09:16.588876+0000","last_peered":"2024-09-21T02:09:16.588876+0000","last_clean":"2024-09-21T02:09:16.588876+0000","last_became_active":"2024-09-21T02:09:15.612114+0000","last_became_peered":"2024-09-21T02:09:15.612114+0000","last_unstale":"2024-09-21T02:09:16.588876+0000","last_undegraded":"2024-09-21T02:09:16.588876+0000","last_fullsized":"2024-09-21T02:09:16.588876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590737+0000","last_change":"2024-09-21T02:09:10.523298+0000","last_active":"2024-09-21T02:09:16.590737+0000","last_peered":"2024-09-21T02:09:16.590737+0000","last_clean":"2024-09-21T02:09:16.590737+0000","last_became_active":"2024-09-21T02:09:10.521768+0000","last_became_peered":"2024-09-21T02:09:10.521768+0000","last_unstale":"2024-09-21T02:09:16.590737+0000","last_undegraded":"2024-09-21T02:09:16.590737+0000","last_fullsized":"2024-09-21T02:09:16.590737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723214+0000","last_change":"2024-09-21T02:09:11.573485+0000","last_active":"2024-09-21T02:09:16.723214+0000","last_peered":"2024-09-21T02:09:16.723214+0000","last_clean":"2024-09-21T02:09:16.723214+0000","last_became_active":"2024-09-21T02:09:11.572854+0000","last_became_peered":"2024-09-21T02:09:11.572854+0000","last_unstale":"2024-09-21T02:09:16.723214+0000","last_undegraded":"2024-09-21T02:09:16.723214+0000","last_fullsized":"2024-09-21T02:09:16.723214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591993+0000","last_change":"2024-09-21T02:09:15.629971+0000","last_active":"2024-09-21T02:09:16.591993+0000","last_peered":"2024-09-21T02:09:16.591993+0000","last_clean":"2024-09-21T02:09:16.591993+0000","last_became_active":"2024-09-21T02:09:15.628916+0000","last_became_peered":"2024-09-21T02:09:15.628916+0000","last_unstale":"2024-09-21T02:09:16.591993+0000","last_undegraded":"2024-09-21T02:09:16.591993+0000","last_fullsized":"2024-09-21T02:09:16.591993+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:16.541985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592632+0000","last_change":"2024-09-21T02:09:10.549868+0000","last_active":"2024-09-21T02:09:16.592632+0000","last_peered":"2024-09-21T02:09:16.592632+0000","last_clean":"2024-09-21T02:09:16.592632+0000","last_became_active":"2024-09-21T02:09:10.549154+0000","last_became_peered":"2024-09-21T02:09:10.549154+0000","last_unstale":"2024-09-21T02:09:16.592632+0000","last_undegraded":"2024-09-21T02:09:16.592632+0000","last_fullsized":"2024-09-21T02:09:16.592632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:42.470809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723996+0000","last_change":"2024-09-21T02:09:11.556559+0000","last_active":"2024-09-21T02:09:16.723996+0000","last_peered":"2024-09-21T02:09:16.723996+0000","last_clean":"2024-09-21T02:09:16.723996+0000","last_became_active":"2024-09-21T02:09:11.555720+0000","last_became_peered":"2024-09-21T02:09:11.555720+0000","last_unstale":"2024-09-21T02:09:16.723996+0000","last_undegraded":"2024-09-21T02:09:16.723996+0000","last_fullsized":"2024-09-21T02:09:16.723996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:43.740023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596031+0000","last_change":"2024-09-21T02:09:15.626432+0000","last_active":"2024-09-21T02:09:16.596031+0000","last_peered":"2024-09-21T02:09:16.596031+0000","last_clean":"2024-09-21T02:09:16.596031+0000","last_became_active":"2024-09-21T02:09:15.620746+0000","last_became_peered":"2024-09-21T02:09:15.620746+0000","last_unstale":"2024-09-21T02:09:16.596031+0000","last_undegraded":"2024-09-21T02:09:16.596031+0000","last_fullsized":"2024-09-21T02:09:16.596031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:46.776489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591909+0000","last_change":"2024-09-21T02:09:10.552140+0000","last_active":"2024-09-21T02:09:16.591909+0000","last_peered":"2024-09-21T02:09:16.591909+0000","last_clean":"2024-09-21T02:09:16.591909+0000","last_became_active":"2024-09-21T02:09:10.551607+0000","last_became_peered":"2024-09-21T02:09:10.551607+0000","last_unstale":"2024-09-21T02:09:16.591909+0000","last_undegraded":"2024-09-21T02:09:16.591909+0000","last_fullsized":"2024-09-21T02:09:16.591909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724056+0000","last_change":"2024-09-21T02:09:11.557870+0000","last_active":"2024-09-21T02:09:16.724056+0000","last_peered":"2024-09-21T02:09:16.724056+0000","last_clean":"2024-09-21T02:09:16.724056+0000","last_became_active":"2024-09-21T02:09:11.556258+0000","last_became_peered":"2024-09-21T02:09:11.556258+0000","last_unstale":"2024-09-21T02:09:16.724056+0000","last_undegraded":"2024-09-21T02:09:16.724056+0000","last_fullsized":"2024-09-21T02:09:16.724056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592325+0000","last_change":"2024-09-21T02:09:15.639499+0000","last_active":"2024-09-21T02:09:16.592325+0000","last_peered":"2024-09-21T02:09:16.592325+0000","last_clean":"2024-09-21T02:09:16.592325+0000","last_became_active":"2024-09-21T02:09:15.638499+0000","last_became_peered":"2024-09-21T02:09:15.638499+0000","last_unstale":"2024-09-21T02:09:16.592325+0000","last_undegraded":"2024-09-21T02:09:16.592325+0000","last_fullsized":"2024-09-21T02:09:16.592325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:22.032011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722630+0000","last_change":"2024-09-21T02:09:11.561488+0000","last_active":"2024-09-21T02:09:16.722630+0000","last_peered":"2024-09-21T02:09:16.722630+0000","last_clean":"2024-09-21T02:09:16.722630+0000","last_became_active":"2024-09-21T02:09:11.560683+0000","last_became_peered":"2024-09-21T02:09:11.560683+0000","last_unstale":"2024-09-21T02:09:16.722630+0000","last_undegraded":"2024-09-21T02:09:16.722630+0000","last_fullsized":"2024-09-21T02:09:16.722630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592616+0000","last_change":"2024-09-21T02:09:10.552547+0000","last_active":"2024-09-21T02:09:16.592616+0000","last_peered":"2024-09-21T02:09:16.592616+0000","last_clean":"2024-09-21T02:09:16.592616+0000","last_became_active":"2024-09-21T02:09:10.552056+0000","last_became_peered":"2024-09-21T02:09:10.552056+0000","last_unstale":"2024-09-21T02:09:16.592616+0000","last_undegraded":"2024-09-21T02:09:16.592616+0000","last_fullsized":"2024-09-21T02:09:16.592616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594025+0000","last_change":"2024-09-21T02:09:15.611360+0000","last_active":"2024-09-21T02:09:16.594025+0000","last_peered":"2024-09-21T02:09:16.594025+0000","last_clean":"2024-09-21T02:09:16.594025+0000","last_became_active":"2024-09-21T02:09:15.609991+0000","last_became_peered":"2024-09-21T02:09:15.609991+0000","last_unstale":"2024-09-21T02:09:16.594025+0000","last_undegraded":"2024-09-21T02:09:16.594025+0000","last_fullsized":"2024-09-21T02:09:16.594025+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:24.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589470+0000","last_change":"2024-09-21T02:09:11.545847+0000","last_active":"2024-09-21T02:09:16.589470+0000","last_peered":"2024-09-21T02:09:16.589470+0000","last_clean":"2024-09-21T02:09:16.589470+0000","last_became_active":"2024-09-21T02:09:11.543606+0000","last_became_peered":"2024-09-21T02:09:11.543606+0000","last_unstale":"2024-09-21T02:09:16.589470+0000","last_undegraded":"2024-09-21T02:09:16.589470+0000","last_fullsized":"2024-09-21T02:09:16.589470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:39.462468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720543+0000","last_change":"2024-09-21T02:09:10.524519+0000","last_active":"2024-09-21T02:09:16.720543+0000","last_peered":"2024-09-21T02:09:16.720543+0000","last_clean":"2024-09-21T02:09:16.720543+0000","last_became_active":"2024-09-21T02:09:10.524074+0000","last_became_peered":"2024-09-21T02:09:10.524074+0000","last_unstale":"2024-09-21T02:09:16.720543+0000","last_undegraded":"2024-09-21T02:09:16.720543+0000","last_fullsized":"2024-09-21T02:09:16.720543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594281+0000","last_change":"2024-09-21T02:09:15.618974+0000","last_active":"2024-09-21T02:09:16.594281+0000","last_peered":"2024-09-21T02:09:16.594281+0000","last_clean":"2024-09-21T02:09:16.594281+0000","last_became_active":"2024-09-21T02:09:15.617774+0000","last_became_peered":"2024-09-21T02:09:15.617774+0000","last_unstale":"2024-09-21T02:09:16.594281+0000","last_undegraded":"2024-09-21T02:09:16.594281+0000","last_fullsized":"2024-09-21T02:09:16.594281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:47.809682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589087+0000","last_change":"2024-09-21T02:09:11.538383+0000","last_active":"2024-09-21T02:09:16.589087+0000","last_peered":"2024-09-21T02:09:16.589087+0000","last_clean":"2024-09-21T02:09:16.589087+0000","last_became_active":"2024-09-21T02:09:11.537569+0000","last_became_peered":"2024-09-21T02:09:11.537569+0000","last_unstale":"2024-09-21T02:09:16.589087+0000","last_undegraded":"2024-09-21T02:09:16.589087+0000","last_fullsized":"2024-09-21T02:09:16.589087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720139+0000","last_change":"2024-09-21T02:09:10.537296+0000","last_active":"2024-09-21T02:09:16.720139+0000","last_peered":"2024-09-21T02:09:16.720139+0000","last_clean":"2024-09-21T02:09:16.720139+0000","last_became_active":"2024-09-21T02:09:10.536913+0000","last_became_peered":"2024-09-21T02:09:10.536913+0000","last_unstale":"2024-09-21T02:09:16.720139+0000","last_undegraded":"2024-09-21T02:09:16.720139+0000","last_fullsized":"2024-09-21T02:09:16.720139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720276+0000","last_change":"2024-09-21T02:09:15.606253+0000","last_active":"2024-09-21T02:09:16.720276+0000","last_peered":"2024-09-21T02:09:16.720276+0000","last_clean":"2024-09-21T02:09:16.720276+0000","last_became_active":"2024-09-21T02:09:15.605847+0000","last_became_peered":"2024-09-21T02:09:15.605847+0000","last_unstale":"2024-09-21T02:09:16.720276+0000","last_undegraded":"2024-09-21T02:09:16.720276+0000","last_fullsized":"2024-09-21T02:09:16.720276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.910212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594287+0000","last_change":"2024-09-21T02:09:11.541952+0000","last_active":"2024-09-21T02:09:16.594287+0000","last_peered":"2024-09-21T02:09:16.594287+0000","last_clean":"2024-09-21T02:09:16.594287+0000","last_became_active":"2024-09-21T02:09:11.540560+0000","last_became_peered":"2024-09-21T02:09:11.540560+0000","last_unstale":"2024-09-21T02:09:16.594287+0000","last_undegraded":"2024-09-21T02:09:16.594287+0000","last_fullsized":"2024-09-21T02:09:16.594287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:25.306228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591648+0000","last_change":"2024-09-21T02:09:10.544405+0000","last_active":"2024-09-21T02:09:16.591648+0000","last_peered":"2024-09-21T02:09:16.591648+0000","last_clean":"2024-09-21T02:09:16.591648+0000","last_became_active":"2024-09-21T02:09:10.540771+0000","last_became_peered":"2024-09-21T02:09:10.540771+0000","last_unstale":"2024-09-21T02:09:16.591648+0000","last_undegraded":"2024-09-21T02:09:16.591648+0000","last_fullsized":"2024-09-21T02:09:16.591648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593729+0000","last_change":"2024-09-21T02:09:15.626770+0000","last_active":"2024-09-21T02:09:16.593729+0000","last_peered":"2024-09-21T02:09:16.593729+0000","last_clean":"2024-09-21T02:09:16.593729+0000","last_became_active":"2024-09-21T02:09:15.623532+0000","last_became_peered":"2024-09-21T02:09:15.623532+0000","last_unstale":"2024-09-21T02:09:16.593729+0000","last_undegraded":"2024-09-21T02:09:16.593729+0000","last_fullsized":"2024-09-21T02:09:16.593729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:52.883467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722937+0000","last_change":"2024-09-21T02:09:11.539618+0000","last_active":"2024-09-21T02:09:16.722937+0000","last_peered":"2024-09-21T02:09:16.722937+0000","last_clean":"2024-09-21T02:09:16.722937+0000","last_became_active":"2024-09-21T02:09:11.538801+0000","last_became_peered":"2024-09-21T02:09:11.538801+0000","last_unstale":"2024-09-21T02:09:16.722937+0000","last_undegraded":"2024-09-21T02:09:16.722937+0000","last_fullsized":"2024-09-21T02:09:16.722937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726772+0000","last_change":"2024-09-21T02:09:10.545874+0000","last_active":"2024-09-21T02:09:16.726772+0000","last_peered":"2024-09-21T02:09:16.726772+0000","last_clean":"2024-09-21T02:09:16.726772+0000","last_became_active":"2024-09-21T02:09:10.545200+0000","last_became_peered":"2024-09-21T02:09:10.545200+0000","last_unstale":"2024-09-21T02:09:16.726772+0000","last_undegraded":"2024-09-21T02:09:16.726772+0000","last_fullsized":"2024-09-21T02:09:16.726772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593400+0000","last_change":"2024-09-21T02:09:15.630472+0000","last_active":"2024-09-21T02:09:16.593400+0000","last_peered":"2024-09-21T02:09:16.593400+0000","last_clean":"2024-09-21T02:09:16.593400+0000","last_became_active":"2024-09-21T02:09:15.628764+0000","last_became_peered":"2024-09-21T02:09:15.628764+0000","last_unstale":"2024-09-21T02:09:16.593400+0000","last_undegraded":"2024-09-21T02:09:16.593400+0000","last_fullsized":"2024-09-21T02:09:16.593400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:21.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591110+0000","last_change":"2024-09-21T02:09:11.543344+0000","last_active":"2024-09-21T02:09:16.591110+0000","last_peered":"2024-09-21T02:09:16.591110+0000","last_clean":"2024-09-21T02:09:16.591110+0000","last_became_active":"2024-09-21T02:09:11.542450+0000","last_became_peered":"2024-09-21T02:09:11.542450+0000","last_unstale":"2024-09-21T02:09:16.591110+0000","last_undegraded":"2024-09-21T02:09:16.591110+0000","last_fullsized":"2024-09-21T02:09:16.591110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:49.206820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719673+0000","last_change":"2024-09-21T02:09:10.541105+0000","last_active":"2024-09-21T02:09:16.719673+0000","last_peered":"2024-09-21T02:09:16.719673+0000","last_clean":"2024-09-21T02:09:16.719673+0000","last_became_active":"2024-09-21T02:09:10.540744+0000","last_became_peered":"2024-09-21T02:09:10.540744+0000","last_unstale":"2024-09-21T02:09:16.719673+0000","last_undegraded":"2024-09-21T02:09:16.719673+0000","last_fullsized":"2024-09-21T02:09:16.719673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:02.568801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588758+0000","last_change":"2024-09-21T02:09:15.613004+0000","last_active":"2024-09-21T02:09:16.588758+0000","last_peered":"2024-09-21T02:09:16.588758+0000","last_clean":"2024-09-21T02:09:16.588758+0000","last_became_active":"2024-09-21T02:09:15.612265+0000","last_became_peered":"2024-09-21T02:09:15.612265+0000","last_unstale":"2024-09-21T02:09:16.588758+0000","last_undegraded":"2024-09-21T02:09:16.588758+0000","last_fullsized":"2024-09-21T02:09:16.588758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721841+0000","last_change":"2024-09-21T02:09:11.528642+0000","last_active":"2024-09-21T02:09:16.721841+0000","last_peered":"2024-09-21T02:09:16.721841+0000","last_clean":"2024-09-21T02:09:16.721841+0000","last_became_active":"2024-09-21T02:09:11.526472+0000","last_became_peered":"2024-09-21T02:09:11.526472+0000","last_unstale":"2024-09-21T02:09:16.721841+0000","last_undegraded":"2024-09-21T02:09:16.721841+0000","last_fullsized":"2024-09-21T02:09:16.721841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593724+0000","last_change":"2024-09-21T02:09:10.553157+0000","last_active":"2024-09-21T02:09:16.593724+0000","last_peered":"2024-09-21T02:09:16.593724+0000","last_clean":"2024-09-21T02:09:16.593724+0000","last_became_active":"2024-09-21T02:09:10.552668+0000","last_became_peered":"2024-09-21T02:09:10.552668+0000","last_unstale":"2024-09-21T02:09:16.593724+0000","last_undegraded":"2024-09-21T02:09:16.593724+0000","last_fullsized":"2024-09-21T02:09:16.593724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:02.667450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723243+0000","last_change":"2024-09-21T02:09:15.626993+0000","last_active":"2024-09-21T02:09:16.723243+0000","last_peered":"2024-09-21T02:09:16.723243+0000","last_clean":"2024-09-21T02:09:16.723243+0000","last_became_active":"2024-09-21T02:09:15.626367+0000","last_became_peered":"2024-09-21T02:09:15.626367+0000","last_unstale":"2024-09-21T02:09:16.723243+0000","last_undegraded":"2024-09-21T02:09:16.723243+0000","last_fullsized":"2024-09-21T02:09:16.723243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:31.903642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724350+0000","last_change":"2024-09-21T02:09:11.557042+0000","last_active":"2024-09-21T02:09:16.724350+0000","last_peered":"2024-09-21T02:09:16.724350+0000","last_clean":"2024-09-21T02:09:16.724350+0000","last_became_active":"2024-09-21T02:09:11.555201+0000","last_became_peered":"2024-09-21T02:09:11.555201+0000","last_unstale":"2024-09-21T02:09:16.724350+0000","last_undegraded":"2024-09-21T02:09:16.724350+0000","last_fullsized":"2024-09-21T02:09:16.724350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:17.236145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722129+0000","last_change":"2024-09-21T02:09:10.516313+0000","last_active":"2024-09-21T02:09:16.722129+0000","last_peered":"2024-09-21T02:09:16.722129+0000","last_clean":"2024-09-21T02:09:16.722129+0000","last_became_active":"2024-09-21T02:09:10.515863+0000","last_became_peered":"2024-09-21T02:09:10.515863+0000","last_unstale":"2024-09-21T02:09:16.722129+0000","last_undegraded":"2024-09-21T02:09:16.722129+0000","last_fullsized":"2024-09-21T02:09:16.722129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:02.067917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719183+0000","last_change":"2024-09-21T02:09:15.624503+0000","last_active":"2024-09-21T02:09:16.719183+0000","last_peered":"2024-09-21T02:09:16.719183+0000","last_clean":"2024-09-21T02:09:16.719183+0000","last_became_active":"2024-09-21T02:09:15.622674+0000","last_became_peered":"2024-09-21T02:09:15.622674+0000","last_unstale":"2024-09-21T02:09:16.719183+0000","last_undegraded":"2024-09-21T02:09:16.719183+0000","last_fullsized":"2024-09-21T02:09:16.719183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:32.890032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.596034+0000","last_change":"2024-09-21T02:09:11.550090+0000","last_active":"2024-09-21T02:09:16.596034+0000","last_peered":"2024-09-21T02:09:16.596034+0000","last_clean":"2024-09-21T02:09:16.596034+0000","last_became_active":"2024-09-21T02:09:11.549246+0000","last_became_peered":"2024-09-21T02:09:11.549246+0000","last_unstale":"2024-09-21T02:09:16.596034+0000","last_undegraded":"2024-09-21T02:09:16.596034+0000","last_fullsized":"2024-09-21T02:09:16.596034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.842699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725146+0000","last_change":"2024-09-21T02:09:10.545849+0000","last_active":"2024-09-21T02:09:16.725146+0000","last_peered":"2024-09-21T02:09:16.725146+0000","last_clean":"2024-09-21T02:09:16.725146+0000","last_became_active":"2024-09-21T02:09:10.544961+0000","last_became_peered":"2024-09-21T02:09:10.544961+0000","last_unstale":"2024-09-21T02:09:16.725146+0000","last_undegraded":"2024-09-21T02:09:16.725146+0000","last_fullsized":"2024-09-21T02:09:16.725146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:43.073819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721335+0000","last_change":"2024-09-21T02:09:15.611136+0000","last_active":"2024-09-21T02:09:16.721335+0000","last_peered":"2024-09-21T02:09:16.721335+0000","last_clean":"2024-09-21T02:09:16.721335+0000","last_became_active":"2024-09-21T02:09:15.610710+0000","last_became_peered":"2024-09-21T02:09:15.610710+0000","last_unstale":"2024-09-21T02:09:16.721335+0000","last_undegraded":"2024-09-21T02:09:16.721335+0000","last_fullsized":"2024-09-21T02:09:16.721335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:11.070735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719495+0000","last_change":"2024-09-21T02:09:11.526818+0000","last_active":"2024-09-21T02:09:16.719495+0000","last_peered":"2024-09-21T02:09:16.719495+0000","last_clean":"2024-09-21T02:09:16.719495+0000","last_became_active":"2024-09-21T02:09:11.525899+0000","last_became_peered":"2024-09-21T02:09:11.525899+0000","last_unstale":"2024-09-21T02:09:16.719495+0000","last_undegraded":"2024-09-21T02:09:16.719495+0000","last_fullsized":"2024-09-21T02:09:16.719495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720218+0000","last_change":"2024-09-21T02:09:10.526158+0000","last_active":"2024-09-21T02:09:16.720218+0000","last_peered":"2024-09-21T02:09:16.720218+0000","last_clean":"2024-09-21T02:09:16.720218+0000","last_became_active":"2024-09-21T02:09:10.525593+0000","last_became_peered":"2024-09-21T02:09:10.525593+0000","last_unstale":"2024-09-21T02:09:16.720218+0000","last_undegraded":"2024-09-21T02:09:16.720218+0000","last_fullsized":"2024-09-21T02:09:16.720218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720063+0000","last_change":"2024-09-21T02:09:15.623852+0000","last_active":"2024-09-21T02:09:16.720063+0000","last_peered":"2024-09-21T02:09:16.720063+0000","last_clean":"2024-09-21T02:09:16.720063+0000","last_became_active":"2024-09-21T02:09:15.623080+0000","last_became_peered":"2024-09-21T02:09:15.623080+0000","last_unstale":"2024-09-21T02:09:16.720063+0000","last_undegraded":"2024-09-21T02:09:16.720063+0000","last_fullsized":"2024-09-21T02:09:16.720063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:38.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722226+0000","last_change":"2024-09-21T02:09:11.558074+0000","last_active":"2024-09-21T02:09:16.722226+0000","last_peered":"2024-09-21T02:09:16.722226+0000","last_clean":"2024-09-21T02:09:16.722226+0000","last_became_active":"2024-09-21T02:09:11.557270+0000","last_became_peered":"2024-09-21T02:09:11.557270+0000","last_unstale":"2024-09-21T02:09:16.722226+0000","last_undegraded":"2024-09-21T02:09:16.722226+0000","last_fullsized":"2024-09-21T02:09:16.722226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:29.728033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590847+0000","last_change":"2024-09-21T02:09:10.540346+0000","last_active":"2024-09-21T02:09:16.590847+0000","last_peered":"2024-09-21T02:09:16.590847+0000","last_clean":"2024-09-21T02:09:16.590847+0000","last_became_active":"2024-09-21T02:09:10.539632+0000","last_became_peered":"2024-09-21T02:09:10.539632+0000","last_unstale":"2024-09-21T02:09:16.590847+0000","last_undegraded":"2024-09-21T02:09:16.590847+0000","last_fullsized":"2024-09-21T02:09:16.590847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:38.681258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722704+0000","last_change":"2024-09-21T02:09:15.636027+0000","last_active":"2024-09-21T02:09:16.722704+0000","last_peered":"2024-09-21T02:09:16.722704+0000","last_clean":"2024-09-21T02:09:16.722704+0000","last_became_active":"2024-09-21T02:09:15.635644+0000","last_became_peered":"2024-09-21T02:09:15.635644+0000","last_unstale":"2024-09-21T02:09:16.722704+0000","last_undegraded":"2024-09-21T02:09:16.722704+0000","last_fullsized":"2024-09-21T02:09:16.722704+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:46.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589275+0000","last_change":"2024-09-21T02:09:11.568395+0000","last_active":"2024-09-21T02:09:16.589275+0000","last_peered":"2024-09-21T02:09:16.589275+0000","last_clean":"2024-09-21T02:09:16.589275+0000","last_became_active":"2024-09-21T02:09:11.567951+0000","last_became_peered":"2024-09-21T02:09:11.567951+0000","last_unstale":"2024-09-21T02:09:16.589275+0000","last_undegraded":"2024-09-21T02:09:16.589275+0000","last_fullsized":"2024-09-21T02:09:16.589275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:38.502107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724904+0000","last_change":"2024-09-21T02:09:10.542137+0000","last_active":"2024-09-21T02:09:16.724904+0000","last_peered":"2024-09-21T02:09:16.724904+0000","last_clean":"2024-09-21T02:09:16.724904+0000","last_became_active":"2024-09-21T02:09:10.541324+0000","last_became_peered":"2024-09-21T02:09:10.541324+0000","last_unstale":"2024-09-21T02:09:16.724904+0000","last_undegraded":"2024-09-21T02:09:16.724904+0000","last_fullsized":"2024-09-21T02:09:16.724904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591215+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:09:16.591215+0000","last_peered":"2024-09-21T02:09:16.591215+0000","last_clean":"2024-09-21T02:09:16.591215+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:09:16.591215+0000","last_undegraded":"2024-09-21T02:09:16.591215+0000","last_fullsized":"2024-09-21T02:09:16.591215+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725602+0000","last_change":"2024-09-21T02:09:15.613320+0000","last_active":"2024-09-21T02:09:16.725602+0000","last_peered":"2024-09-21T02:09:16.725602+0000","last_clean":"2024-09-21T02:09:16.725602+0000","last_became_active":"2024-09-21T02:09:15.612050+0000","last_became_peered":"2024-09-21T02:09:15.612050+0000","last_unstale":"2024-09-21T02:09:16.725602+0000","last_undegraded":"2024-09-21T02:09:16.725602+0000","last_fullsized":"2024-09-21T02:09:16.725602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:54.519499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592039+0000","last_change":"2024-09-21T02:09:11.571228+0000","last_active":"2024-09-21T02:09:16.592039+0000","last_peered":"2024-09-21T02:09:16.592039+0000","last_clean":"2024-09-21T02:09:16.592039+0000","last_became_active":"2024-09-21T02:09:11.570627+0000","last_became_peered":"2024-09-21T02:09:11.570627+0000","last_unstale":"2024-09-21T02:09:16.592039+0000","last_undegraded":"2024-09-21T02:09:16.592039+0000","last_fullsized":"2024-09-21T02:09:16.592039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.513856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721976+0000","last_change":"2024-09-21T02:09:10.530153+0000","last_active":"2024-09-21T02:09:16.721976+0000","last_peered":"2024-09-21T02:09:16.721976+0000","last_clean":"2024-09-21T02:09:16.721976+0000","last_became_active":"2024-09-21T02:09:10.529650+0000","last_became_peered":"2024-09-21T02:09:10.529650+0000","last_unstale":"2024-09-21T02:09:16.721976+0000","last_undegraded":"2024-09-21T02:09:16.721976+0000","last_fullsized":"2024-09-21T02:09:16.721976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:41.830907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724037+0000","last_change":"2024-09-21T02:09:15.609642+0000","last_active":"2024-09-21T02:09:16.724037+0000","last_peered":"2024-09-21T02:09:16.724037+0000","last_clean":"2024-09-21T02:09:16.724037+0000","last_became_active":"2024-09-21T02:09:15.604449+0000","last_became_peered":"2024-09-21T02:09:15.604449+0000","last_unstale":"2024-09-21T02:09:16.724037+0000","last_undegraded":"2024-09-21T02:09:16.724037+0000","last_fullsized":"2024-09-21T02:09:16.724037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:55.894914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724291+0000","last_change":"2024-09-21T02:09:11.554539+0000","last_active":"2024-09-21T02:09:16.724291+0000","last_peered":"2024-09-21T02:09:16.724291+0000","last_clean":"2024-09-21T02:09:16.724291+0000","last_became_active":"2024-09-21T02:09:11.553712+0000","last_became_peered":"2024-09-21T02:09:11.553712+0000","last_unstale":"2024-09-21T02:09:16.724291+0000","last_undegraded":"2024-09-21T02:09:16.724291+0000","last_fullsized":"2024-09-21T02:09:16.724291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.229624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719880+0000","last_change":"2024-09-21T02:09:10.545134+0000","last_active":"2024-09-21T02:09:16.719880+0000","last_peered":"2024-09-21T02:09:16.719880+0000","last_clean":"2024-09-21T02:09:16.719880+0000","last_became_active":"2024-09-21T02:09:10.544792+0000","last_became_peered":"2024-09-21T02:09:10.544792+0000","last_unstale":"2024-09-21T02:09:16.719880+0000","last_undegraded":"2024-09-21T02:09:16.719880+0000","last_fullsized":"2024-09-21T02:09:16.719880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:52.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723145+0000","last_change":"2024-09-21T02:09:15.626483+0000","last_active":"2024-09-21T02:09:16.723145+0000","last_peered":"2024-09-21T02:09:16.723145+0000","last_clean":"2024-09-21T02:09:16.723145+0000","last_became_active":"2024-09-21T02:09:15.626132+0000","last_became_peered":"2024-09-21T02:09:15.626132+0000","last_unstale":"2024-09-21T02:09:16.723145+0000","last_undegraded":"2024-09-21T02:09:16.723145+0000","last_fullsized":"2024-09-21T02:09:16.723145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:04.011318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592725+0000","last_change":"2024-09-21T02:09:11.533777+0000","last_active":"2024-09-21T02:09:16.592725+0000","last_peered":"2024-09-21T02:09:16.592725+0000","last_clean":"2024-09-21T02:09:16.592725+0000","last_became_active":"2024-09-21T02:09:11.532928+0000","last_became_peered":"2024-09-21T02:09:11.532928+0000","last_unstale":"2024-09-21T02:09:16.592725+0000","last_undegraded":"2024-09-21T02:09:16.592725+0000","last_fullsized":"2024-09-21T02:09:16.592725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:21.176822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721695+0000","last_change":"2024-09-21T02:09:10.524899+0000","last_active":"2024-09-21T02:09:16.721695+0000","last_peered":"2024-09-21T02:09:16.721695+0000","last_clean":"2024-09-21T02:09:16.721695+0000","last_became_active":"2024-09-21T02:09:10.524447+0000","last_became_peered":"2024-09-21T02:09:10.524447+0000","last_unstale":"2024-09-21T02:09:16.721695+0000","last_undegraded":"2024-09-21T02:09:16.721695+0000","last_fullsized":"2024-09-21T02:09:16.721695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723182+0000","last_change":"2024-09-21T02:09:15.623410+0000","last_active":"2024-09-21T02:09:16.723182+0000","last_peered":"2024-09-21T02:09:16.723182+0000","last_clean":"2024-09-21T02:09:16.723182+0000","last_became_active":"2024-09-21T02:09:15.622194+0000","last_became_peered":"2024-09-21T02:09:15.622194+0000","last_unstale":"2024-09-21T02:09:16.723182+0000","last_undegraded":"2024-09-21T02:09:16.723182+0000","last_fullsized":"2024-09-21T02:09:16.723182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:01.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594437+0000","last_change":"2024-09-21T02:09:11.531628+0000","last_active":"2024-09-21T02:09:16.594437+0000","last_peered":"2024-09-21T02:09:16.594437+0000","last_clean":"2024-09-21T02:09:16.594437+0000","last_became_active":"2024-09-21T02:09:11.522487+0000","last_became_peered":"2024-09-21T02:09:11.522487+0000","last_unstale":"2024-09-21T02:09:16.594437+0000","last_undegraded":"2024-09-21T02:09:16.594437+0000","last_fullsized":"2024-09-21T02:09:16.594437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:31.403053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595653+0000","last_change":"2024-09-21T02:09:10.526000+0000","last_active":"2024-09-21T02:09:16.595653+0000","last_peered":"2024-09-21T02:09:16.595653+0000","last_clean":"2024-09-21T02:09:16.595653+0000","last_became_active":"2024-09-21T02:09:10.525267+0000","last_became_peered":"2024-09-21T02:09:10.525267+0000","last_unstale":"2024-09-21T02:09:16.595653+0000","last_undegraded":"2024-09-21T02:09:16.595653+0000","last_fullsized":"2024-09-21T02:09:16.595653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:32.448144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592641+0000","last_change":"2024-09-21T02:09:15.626495+0000","last_active":"2024-09-21T02:09:16.592641+0000","last_peered":"2024-09-21T02:09:16.592641+0000","last_clean":"2024-09-21T02:09:16.592641+0000","last_became_active":"2024-09-21T02:09:15.625151+0000","last_became_peered":"2024-09-21T02:09:15.625151+0000","last_unstale":"2024-09-21T02:09:16.592641+0000","last_undegraded":"2024-09-21T02:09:16.592641+0000","last_fullsized":"2024-09-21T02:09:16.592641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:23.340983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591341+0000","last_change":"2024-09-21T02:09:11.521241+0000","last_active":"2024-09-21T02:09:16.591341+0000","last_peered":"2024-09-21T02:09:16.591341+0000","last_clean":"2024-09-21T02:09:16.591341+0000","last_became_active":"2024-09-21T02:09:11.519883+0000","last_became_peered":"2024-09-21T02:09:11.519883+0000","last_unstale":"2024-09-21T02:09:16.591341+0000","last_undegraded":"2024-09-21T02:09:16.591341+0000","last_fullsized":"2024-09-21T02:09:16.591341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:26.260780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720026+0000","last_change":"2024-09-21T02:09:10.520638+0000","last_active":"2024-09-21T02:09:16.720026+0000","last_peered":"2024-09-21T02:09:16.720026+0000","last_clean":"2024-09-21T02:09:16.720026+0000","last_became_active":"2024-09-21T02:09:10.520131+0000","last_became_peered":"2024-09-21T02:09:10.520131+0000","last_unstale":"2024-09-21T02:09:16.720026+0000","last_undegraded":"2024-09-21T02:09:16.720026+0000","last_fullsized":"2024-09-21T02:09:16.720026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723034+0000","last_change":"2024-09-21T02:09:15.610523+0000","last_active":"2024-09-21T02:09:16.723034+0000","last_peered":"2024-09-21T02:09:16.723034+0000","last_clean":"2024-09-21T02:09:16.723034+0000","last_became_active":"2024-09-21T02:09:15.609441+0000","last_became_peered":"2024-09-21T02:09:15.609441+0000","last_unstale":"2024-09-21T02:09:16.723034+0000","last_undegraded":"2024-09-21T02:09:16.723034+0000","last_fullsized":"2024-09-21T02:09:16.723034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:36.011939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722864+0000","last_change":"2024-09-21T02:09:11.561816+0000","last_active":"2024-09-21T02:09:16.722864+0000","last_peered":"2024-09-21T02:09:16.722864+0000","last_clean":"2024-09-21T02:09:16.722864+0000","last_became_active":"2024-09-21T02:09:11.561231+0000","last_became_peered":"2024-09-21T02:09:11.561231+0000","last_unstale":"2024-09-21T02:09:16.722864+0000","last_undegraded":"2024-09-21T02:09:16.722864+0000","last_fullsized":"2024-09-21T02:09:16.722864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:38.366733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723173+0000","last_change":"2024-09-21T02:09:10.550508+0000","last_active":"2024-09-21T02:09:16.723173+0000","last_peered":"2024-09-21T02:09:16.723173+0000","last_clean":"2024-09-21T02:09:16.723173+0000","last_became_active":"2024-09-21T02:09:10.549913+0000","last_became_peered":"2024-09-21T02:09:10.549913+0000","last_unstale":"2024-09-21T02:09:16.723173+0000","last_undegraded":"2024-09-21T02:09:16.723173+0000","last_fullsized":"2024-09-21T02:09:16.723173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:15.201315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591175+0000","last_change":"2024-09-21T02:09:15.618146+0000","last_active":"2024-09-21T02:09:16.591175+0000","last_peered":"2024-09-21T02:09:16.591175+0000","last_clean":"2024-09-21T02:09:16.591175+0000","last_became_active":"2024-09-21T02:09:15.617195+0000","last_became_peered":"2024-09-21T02:09:15.617195+0000","last_unstale":"2024-09-21T02:09:16.591175+0000","last_undegraded":"2024-09-21T02:09:16.591175+0000","last_fullsized":"2024-09-21T02:09:16.591175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.452372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592327+0000","last_change":"2024-09-21T02:09:11.520992+0000","last_active":"2024-09-21T02:09:16.592327+0000","last_peered":"2024-09-21T02:09:16.592327+0000","last_clean":"2024-09-21T02:09:16.592327+0000","last_became_active":"2024-09-21T02:09:11.519892+0000","last_became_peered":"2024-09-21T02:09:11.519892+0000","last_unstale":"2024-09-21T02:09:16.592327+0000","last_undegraded":"2024-09-21T02:09:16.592327+0000","last_fullsized":"2024-09-21T02:09:16.592327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:34.673193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724869+0000","last_change":"2024-09-21T02:09:10.543820+0000","last_active":"2024-09-21T02:09:16.724869+0000","last_peered":"2024-09-21T02:09:16.724869+0000","last_clean":"2024-09-21T02:09:16.724869+0000","last_became_active":"2024-09-21T02:09:10.542944+0000","last_became_peered":"2024-09-21T02:09:10.542944+0000","last_unstale":"2024-09-21T02:09:16.724869+0000","last_undegraded":"2024-09-21T02:09:16.724869+0000","last_fullsized":"2024-09-21T02:09:16.724869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:59.409264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590058+0000","last_change":"2024-09-21T02:09:15.612341+0000","last_active":"2024-09-21T02:09:16.590058+0000","last_peered":"2024-09-21T02:09:16.590058+0000","last_clean":"2024-09-21T02:09:16.590058+0000","last_became_active":"2024-09-21T02:09:15.611864+0000","last_became_peered":"2024-09-21T02:09:15.611864+0000","last_unstale":"2024-09-21T02:09:16.590058+0000","last_undegraded":"2024-09-21T02:09:16.590058+0000","last_fullsized":"2024-09-21T02:09:16.590058+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:25.484143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723926+0000","last_change":"2024-09-21T02:09:11.557442+0000","last_active":"2024-09-21T02:09:16.723926+0000","last_peered":"2024-09-21T02:09:16.723926+0000","last_clean":"2024-09-21T02:09:16.723926+0000","last_became_active":"2024-09-21T02:09:11.557119+0000","last_became_peered":"2024-09-21T02:09:11.557119+0000","last_unstale":"2024-09-21T02:09:16.723926+0000","last_undegraded":"2024-09-21T02:09:16.723926+0000","last_fullsized":"2024-09-21T02:09:16.723926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:30.951569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723410+0000","last_change":"2024-09-21T02:09:10.530343+0000","last_active":"2024-09-21T02:09:16.723410+0000","last_peered":"2024-09-21T02:09:16.723410+0000","last_clean":"2024-09-21T02:09:16.723410+0000","last_became_active":"2024-09-21T02:09:10.529930+0000","last_became_peered":"2024-09-21T02:09:10.529930+0000","last_unstale":"2024-09-21T02:09:16.723410+0000","last_undegraded":"2024-09-21T02:09:16.723410+0000","last_fullsized":"2024-09-21T02:09:16.723410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.949461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725837+0000","last_change":"2024-09-21T02:09:15.635216+0000","last_active":"2024-09-21T02:09:16.725837+0000","last_peered":"2024-09-21T02:09:16.725837+0000","last_clean":"2024-09-21T02:09:16.725837+0000","last_became_active":"2024-09-21T02:09:15.626937+0000","last_became_peered":"2024-09-21T02:09:15.626937+0000","last_unstale":"2024-09-21T02:09:16.725837+0000","last_undegraded":"2024-09-21T02:09:16.725837+0000","last_fullsized":"2024-09-21T02:09:16.725837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591747+0000","last_change":"2024-09-21T02:09:11.538575+0000","last_active":"2024-09-21T02:09:16.591747+0000","last_peered":"2024-09-21T02:09:16.591747+0000","last_clean":"2024-09-21T02:09:16.591747+0000","last_became_active":"2024-09-21T02:09:11.535561+0000","last_became_peered":"2024-09-21T02:09:11.535561+0000","last_unstale":"2024-09-21T02:09:16.591747+0000","last_undegraded":"2024-09-21T02:09:16.591747+0000","last_fullsized":"2024-09-21T02:09:16.591747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:58.627736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:22.534320+0000","last_change":"2024-09-21T02:09:10.546712+0000","last_active":"2024-09-21T02:09:22.534320+0000","last_peered":"2024-09-21T02:09:22.534320+0000","last_clean":"2024-09-21T02:09:22.534320+0000","last_became_active":"2024-09-21T02:09:10.541968+0000","last_became_peered":"2024-09-21T02:09:10.541968+0000","last_unstale":"2024-09-21T02:09:22.534320+0000","last_undegraded":"2024-09-21T02:09:22.534320+0000","last_fullsized":"2024-09-21T02:09:22.534320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720210+0000","last_change":"2024-09-21T02:09:15.616116+0000","last_active":"2024-09-21T02:09:16.720210+0000","last_peered":"2024-09-21T02:09:16.720210+0000","last_clean":"2024-09-21T02:09:16.720210+0000","last_became_active":"2024-09-21T02:09:15.615337+0000","last_became_peered":"2024-09-21T02:09:15.615337+0000","last_unstale":"2024-09-21T02:09:16.720210+0000","last_undegraded":"2024-09-21T02:09:16.720210+0000","last_fullsized":"2024-09-21T02:09:16.720210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:35.472712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.594788+0000","last_change":"2024-09-21T02:09:11.520271+0000","last_active":"2024-09-21T02:09:16.594788+0000","last_peered":"2024-09-21T02:09:16.594788+0000","last_clean":"2024-09-21T02:09:16.594788+0000","last_became_active":"2024-09-21T02:09:11.518967+0000","last_became_peered":"2024-09-21T02:09:11.518967+0000","last_unstale":"2024-09-21T02:09:16.594788+0000","last_undegraded":"2024-09-21T02:09:16.594788+0000","last_fullsized":"2024-09-21T02:09:16.594788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719861+0000","last_change":"2024-09-21T02:09:10.536568+0000","last_active":"2024-09-21T02:09:16.719861+0000","last_peered":"2024-09-21T02:09:16.719861+0000","last_clean":"2024-09-21T02:09:16.719861+0000","last_became_active":"2024-09-21T02:09:10.536266+0000","last_became_peered":"2024-09-21T02:09:10.536266+0000","last_unstale":"2024-09-21T02:09:16.719861+0000","last_undegraded":"2024-09-21T02:09:16.719861+0000","last_fullsized":"2024-09-21T02:09:16.719861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:19.055651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595233+0000","last_change":"2024-09-21T02:09:15.637262+0000","last_active":"2024-09-21T02:09:16.595233+0000","last_peered":"2024-09-21T02:09:16.595233+0000","last_clean":"2024-09-21T02:09:16.595233+0000","last_became_active":"2024-09-21T02:09:15.636305+0000","last_became_peered":"2024-09-21T02:09:15.636305+0000","last_unstale":"2024-09-21T02:09:16.595233+0000","last_undegraded":"2024-09-21T02:09:16.595233+0000","last_fullsized":"2024-09-21T02:09:16.595233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:36.447874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723352+0000","last_change":"2024-09-21T02:09:11.562979+0000","last_active":"2024-09-21T02:09:16.723352+0000","last_peered":"2024-09-21T02:09:16.723352+0000","last_clean":"2024-09-21T02:09:16.723352+0000","last_became_active":"2024-09-21T02:09:11.559009+0000","last_became_peered":"2024-09-21T02:09:11.559009+0000","last_unstale":"2024-09-21T02:09:16.723352+0000","last_undegraded":"2024-09-21T02:09:16.723352+0000","last_fullsized":"2024-09-21T02:09:16.723352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:27.296832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725417+0000","last_change":"2024-09-21T02:09:10.546521+0000","last_active":"2024-09-21T02:09:16.725417+0000","last_peered":"2024-09-21T02:09:16.725417+0000","last_clean":"2024-09-21T02:09:16.725417+0000","last_became_active":"2024-09-21T02:09:10.545864+0000","last_became_peered":"2024-09-21T02:09:10.545864+0000","last_unstale":"2024-09-21T02:09:16.725417+0000","last_undegraded":"2024-09-21T02:09:16.725417+0000","last_fullsized":"2024-09-21T02:09:16.725417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:43.440149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589335+0000","last_change":"2024-09-21T02:09:15.616179+0000","last_active":"2024-09-21T02:09:16.589335+0000","last_peered":"2024-09-21T02:09:16.589335+0000","last_clean":"2024-09-21T02:09:16.589335+0000","last_became_active":"2024-09-21T02:09:15.615344+0000","last_became_peered":"2024-09-21T02:09:15.615344+0000","last_unstale":"2024-09-21T02:09:16.589335+0000","last_undegraded":"2024-09-21T02:09:16.589335+0000","last_fullsized":"2024-09-21T02:09:16.589335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.519680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725123+0000","last_change":"2024-09-21T02:09:11.582832+0000","last_active":"2024-09-21T02:09:16.725123+0000","last_peered":"2024-09-21T02:09:16.725123+0000","last_clean":"2024-09-21T02:09:16.725123+0000","last_became_active":"2024-09-21T02:09:11.582121+0000","last_became_peered":"2024-09-21T02:09:11.582121+0000","last_unstale":"2024-09-21T02:09:16.725123+0000","last_undegraded":"2024-09-21T02:09:16.725123+0000","last_fullsized":"2024-09-21T02:09:16.725123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:54.829896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726463+0000","last_change":"2024-09-21T02:09:10.556671+0000","last_active":"2024-09-21T02:09:16.726463+0000","last_peered":"2024-09-21T02:09:16.726463+0000","last_clean":"2024-09-21T02:09:16.726463+0000","last_became_active":"2024-09-21T02:09:10.555875+0000","last_became_peered":"2024-09-21T02:09:10.555875+0000","last_unstale":"2024-09-21T02:09:16.726463+0000","last_undegraded":"2024-09-21T02:09:16.726463+0000","last_fullsized":"2024-09-21T02:09:16.726463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:58.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588965+0000","last_change":"2024-09-21T02:09:15.623197+0000","last_active":"2024-09-21T02:09:16.588965+0000","last_peered":"2024-09-21T02:09:16.588965+0000","last_clean":"2024-09-21T02:09:16.588965+0000","last_became_active":"2024-09-21T02:09:15.622340+0000","last_became_peered":"2024-09-21T02:09:15.622340+0000","last_unstale":"2024-09-21T02:09:16.588965+0000","last_undegraded":"2024-09-21T02:09:16.588965+0000","last_fullsized":"2024-09-21T02:09:16.588965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:58.290453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722347+0000","last_change":"2024-09-21T02:09:15.636227+0000","last_active":"2024-09-21T02:09:16.722347+0000","last_peered":"2024-09-21T02:09:16.722347+0000","last_clean":"2024-09-21T02:09:16.722347+0000","last_became_active":"2024-09-21T02:09:15.635814+0000","last_became_peered":"2024-09-21T02:09:15.635814+0000","last_unstale":"2024-09-21T02:09:16.722347+0000","last_undegraded":"2024-09-21T02:09:16.722347+0000","last_fullsized":"2024-09-21T02:09:16.722347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:37.103933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719830+0000","last_change":"2024-09-21T02:09:11.531143+0000","last_active":"2024-09-21T02:09:16.719830+0000","last_peered":"2024-09-21T02:09:16.719830+0000","last_clean":"2024-09-21T02:09:16.719830+0000","last_became_active":"2024-09-21T02:09:11.530348+0000","last_became_peered":"2024-09-21T02:09:11.530348+0000","last_unstale":"2024-09-21T02:09:16.719830+0000","last_undegraded":"2024-09-21T02:09:16.719830+0000","last_fullsized":"2024-09-21T02:09:16.719830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:21.823287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723931+0000","last_change":"2024-09-21T02:09:10.538525+0000","last_active":"2024-09-21T02:09:16.723931+0000","last_peered":"2024-09-21T02:09:16.723931+0000","last_clean":"2024-09-21T02:09:16.723931+0000","last_became_active":"2024-09-21T02:09:10.537466+0000","last_became_peered":"2024-09-21T02:09:10.537466+0000","last_unstale":"2024-09-21T02:09:16.723931+0000","last_undegraded":"2024-09-21T02:09:16.723931+0000","last_fullsized":"2024-09-21T02:09:16.723931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726815+0000","last_change":"2024-09-21T02:09:15.625976+0000","last_active":"2024-09-21T02:09:16.726815+0000","last_peered":"2024-09-21T02:09:16.726815+0000","last_clean":"2024-09-21T02:09:16.726815+0000","last_became_active":"2024-09-21T02:09:15.624692+0000","last_became_peered":"2024-09-21T02:09:15.624692+0000","last_unstale":"2024-09-21T02:09:16.726815+0000","last_undegraded":"2024-09-21T02:09:16.726815+0000","last_fullsized":"2024-09-21T02:09:16.726815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:18.311891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588890+0000","last_change":"2024-09-21T02:09:11.544871+0000","last_active":"2024-09-21T02:09:16.588890+0000","last_peered":"2024-09-21T02:09:16.588890+0000","last_clean":"2024-09-21T02:09:16.588890+0000","last_became_active":"2024-09-21T02:09:11.543403+0000","last_became_peered":"2024-09-21T02:09:11.543403+0000","last_unstale":"2024-09-21T02:09:16.588890+0000","last_undegraded":"2024-09-21T02:09:16.588890+0000","last_fullsized":"2024-09-21T02:09:16.588890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:45.667456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589183+0000","last_change":"2024-09-21T02:09:10.539555+0000","last_active":"2024-09-21T02:09:16.589183+0000","last_peered":"2024-09-21T02:09:16.589183+0000","last_clean":"2024-09-21T02:09:16.589183+0000","last_became_active":"2024-09-21T02:09:10.539163+0000","last_became_peered":"2024-09-21T02:09:10.539163+0000","last_unstale":"2024-09-21T02:09:16.589183+0000","last_undegraded":"2024-09-21T02:09:16.589183+0000","last_fullsized":"2024-09-21T02:09:16.589183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:49.179826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724509+0000","last_change":"2024-09-21T02:09:10.550879+0000","last_active":"2024-09-21T02:09:16.724509+0000","last_peered":"2024-09-21T02:09:16.724509+0000","last_clean":"2024-09-21T02:09:16.724509+0000","last_became_active":"2024-09-21T02:09:10.550147+0000","last_became_peered":"2024-09-21T02:09:10.550147+0000","last_unstale":"2024-09-21T02:09:16.724509+0000","last_undegraded":"2024-09-21T02:09:16.724509+0000","last_fullsized":"2024-09-21T02:09:16.724509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:59.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723322+0000","last_change":"2024-09-21T02:09:11.564520+0000","last_active":"2024-09-21T02:09:16.723322+0000","last_peered":"2024-09-21T02:09:16.723322+0000","last_clean":"2024-09-21T02:09:16.723322+0000","last_became_active":"2024-09-21T02:09:11.563870+0000","last_became_peered":"2024-09-21T02:09:11.563870+0000","last_unstale":"2024-09-21T02:09:16.723322+0000","last_undegraded":"2024-09-21T02:09:16.723322+0000","last_fullsized":"2024-09-21T02:09:16.723322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:20.175775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721646+0000","last_change":"2024-09-21T02:09:15.623267+0000","last_active":"2024-09-21T02:09:16.721646+0000","last_peered":"2024-09-21T02:09:16.721646+0000","last_clean":"2024-09-21T02:09:16.721646+0000","last_became_active":"2024-09-21T02:09:15.622816+0000","last_became_peered":"2024-09-21T02:09:15.622816+0000","last_unstale":"2024-09-21T02:09:16.721646+0000","last_undegraded":"2024-09-21T02:09:16.721646+0000","last_fullsized":"2024-09-21T02:09:16.721646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:11.398708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589624+0000","last_change":"2024-09-21T02:09:10.545988+0000","last_active":"2024-09-21T02:09:16.589624+0000","last_peered":"2024-09-21T02:09:16.589624+0000","last_clean":"2024-09-21T02:09:16.589624+0000","last_became_active":"2024-09-21T02:09:10.545537+0000","last_became_peered":"2024-09-21T02:09:10.545537+0000","last_unstale":"2024-09-21T02:09:16.589624+0000","last_undegraded":"2024-09-21T02:09:16.589624+0000","last_fullsized":"2024-09-21T02:09:16.589624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:09.469930+0000","last_clean_scrub_stamp":"2024-09-21T02:09:09.469930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:09.063264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592924+0000","last_change":"2024-09-21T02:09:11.527921+0000","last_active":"2024-09-21T02:09:16.592924+0000","last_peered":"2024-09-21T02:09:16.592924+0000","last_clean":"2024-09-21T02:09:16.592924+0000","last_became_active":"2024-09-21T02:09:11.527033+0000","last_became_peered":"2024-09-21T02:09:11.527033+0000","last_unstale":"2024-09-21T02:09:16.592924+0000","last_undegraded":"2024-09-21T02:09:16.592924+0000","last_fullsized":"2024-09-21T02:09:16.592924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590282+0000","last_change":"2024-09-21T02:09:15.608837+0000","last_active":"2024-09-21T02:09:16.590282+0000","last_peered":"2024-09-21T02:09:16.590282+0000","last_clean":"2024-09-21T02:09:16.590282+0000","last_became_active":"2024-09-21T02:09:15.607905+0000","last_became_peered":"2024-09-21T02:09:15.607905+0000","last_unstale":"2024-09-21T02:09:16.590282+0000","last_undegraded":"2024-09-21T02:09:16.590282+0000","last_fullsized":"2024-09-21T02:09:16.590282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:14.551381+0000","last_clean_scrub_stamp":"2024-09-21T02:09:14.551381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:15.347636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724603+0000","last_change":"2024-09-21T02:09:11.562756+0000","last_active":"2024-09-21T02:09:16.724603+0000","last_peered":"2024-09-21T02:09:16.724603+0000","last_clean":"2024-09-21T02:09:16.724603+0000","last_became_active":"2024-09-21T02:09:11.560604+0000","last_became_peered":"2024-09-21T02:09:11.560604+0000","last_unstale":"2024-09-21T02:09:16.724603+0000","last_undegraded":"2024-09-21T02:09:16.724603+0000","last_fullsized":"2024-09-21T02:09:16.724603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:27.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719949+0000","last_change":"2024-09-21T02:09:11.566770+0000","last_active":"2024-09-21T02:09:16.719949+0000","last_peered":"2024-09-21T02:09:16.719949+0000","last_clean":"2024-09-21T02:09:16.719949+0000","last_became_active":"2024-09-21T02:09:11.566425+0000","last_became_peered":"2024-09-21T02:09:11.566425+0000","last_unstale":"2024-09-21T02:09:16.719949+0000","last_undegraded":"2024-09-21T02:09:16.719949+0000","last_fullsized":"2024-09-21T02:09:16.719949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:10.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724847+0000","last_change":"2024-09-21T02:09:11.572089+0000","last_active":"2024-09-21T02:09:16.724847+0000","last_peered":"2024-09-21T02:09:16.724847+0000","last_clean":"2024-09-21T02:09:16.724847+0000","last_became_active":"2024-09-21T02:09:11.566576+0000","last_became_peered":"2024-09-21T02:09:11.566576+0000","last_unstale":"2024-09-21T02:09:16.724847+0000","last_undegraded":"2024-09-21T02:09:16.724847+0000","last_fullsized":"2024-09-21T02:09:16.724847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:42.706829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723234+0000","last_change":"2024-09-21T02:09:11.564192+0000","last_active":"2024-09-21T02:09:16.723234+0000","last_peered":"2024-09-21T02:09:16.723234+0000","last_clean":"2024-09-21T02:09:16.723234+0000","last_became_active":"2024-09-21T02:09:11.559079+0000","last_became_peered":"2024-09-21T02:09:11.559079+0000","last_unstale":"2024-09-21T02:09:16.723234+0000","last_undegraded":"2024-09-21T02:09:16.723234+0000","last_fullsized":"2024-09-21T02:09:16.723234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590380+0000","last_change":"2024-09-21T02:09:11.540588+0000","last_active":"2024-09-21T02:09:16.590380+0000","last_peered":"2024-09-21T02:09:16.590380+0000","last_clean":"2024-09-21T02:09:16.590380+0000","last_became_active":"2024-09-21T02:09:11.539779+0000","last_became_peered":"2024-09-21T02:09:11.539779+0000","last_unstale":"2024-09-21T02:09:16.590380+0000","last_undegraded":"2024-09-21T02:09:16.590380+0000","last_fullsized":"2024-09-21T02:09:16.590380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:24.826345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719412+0000","last_change":"2024-09-21T02:09:11.530898+0000","last_active":"2024-09-21T02:09:16.719412+0000","last_peered":"2024-09-21T02:09:16.719412+0000","last_clean":"2024-09-21T02:09:16.719412+0000","last_became_active":"2024-09-21T02:09:11.530100+0000","last_became_peered":"2024-09-21T02:09:11.530100+0000","last_unstale":"2024-09-21T02:09:16.719412+0000","last_undegraded":"2024-09-21T02:09:16.719412+0000","last_fullsized":"2024-09-21T02:09:16.719412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:08.206760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726838+0000","last_change":"2024-09-21T02:09:11.549198+0000","last_active":"2024-09-21T02:09:16.726838+0000","last_peered":"2024-09-21T02:09:16.726838+0000","last_clean":"2024-09-21T02:09:16.726838+0000","last_became_active":"2024-09-21T02:09:11.548421+0000","last_became_peered":"2024-09-21T02:09:11.548421+0000","last_unstale":"2024-09-21T02:09:16.726838+0000","last_undegraded":"2024-09-21T02:09:16.726838+0000","last_fullsized":"2024-09-21T02:09:16.726838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:22.505823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.590900+0000","last_change":"2024-09-21T02:09:11.552426+0000","last_active":"2024-09-21T02:09:16.590900+0000","last_peered":"2024-09-21T02:09:16.590900+0000","last_clean":"2024-09-21T02:09:16.590900+0000","last_became_active":"2024-09-21T02:09:11.551844+0000","last_became_peered":"2024-09-21T02:09:11.551844+0000","last_unstale":"2024-09-21T02:09:16.590900+0000","last_undegraded":"2024-09-21T02:09:16.590900+0000","last_fullsized":"2024-09-21T02:09:16.590900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:45.601375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723635+0000","last_change":"2024-09-21T02:09:11.566549+0000","last_active":"2024-09-21T02:09:16.723635+0000","last_peered":"2024-09-21T02:09:16.723635+0000","last_clean":"2024-09-21T02:09:16.723635+0000","last_became_active":"2024-09-21T02:09:11.565905+0000","last_became_peered":"2024-09-21T02:09:11.565905+0000","last_unstale":"2024-09-21T02:09:16.723635+0000","last_undegraded":"2024-09-21T02:09:16.723635+0000","last_fullsized":"2024-09-21T02:09:16.723635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.460538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589185+0000","last_change":"2024-09-21T02:09:11.552230+0000","last_active":"2024-09-21T02:09:16.589185+0000","last_peered":"2024-09-21T02:09:16.589185+0000","last_clean":"2024-09-21T02:09:16.589185+0000","last_became_active":"2024-09-21T02:09:11.551873+0000","last_became_peered":"2024-09-21T02:09:11.551873+0000","last_unstale":"2024-09-21T02:09:16.589185+0000","last_undegraded":"2024-09-21T02:09:16.589185+0000","last_fullsized":"2024-09-21T02:09:16.589185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724470+0000","last_change":"2024-09-21T02:09:11.581809+0000","last_active":"2024-09-21T02:09:16.724470+0000","last_peered":"2024-09-21T02:09:16.724470+0000","last_clean":"2024-09-21T02:09:16.724470+0000","last_became_active":"2024-09-21T02:09:11.581010+0000","last_became_peered":"2024-09-21T02:09:11.581010+0000","last_unstale":"2024-09-21T02:09:16.724470+0000","last_undegraded":"2024-09-21T02:09:16.724470+0000","last_fullsized":"2024-09-21T02:09:16.724470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:19.271324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722807+0000","last_change":"2024-09-21T02:09:11.558586+0000","last_active":"2024-09-21T02:09:16.722807+0000","last_peered":"2024-09-21T02:09:16.722807+0000","last_clean":"2024-09-21T02:09:16.722807+0000","last_became_active":"2024-09-21T02:09:11.558126+0000","last_became_peered":"2024-09-21T02:09:11.558126+0000","last_unstale":"2024-09-21T02:09:16.722807+0000","last_undegraded":"2024-09-21T02:09:16.722807+0000","last_fullsized":"2024-09-21T02:09:16.722807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:10.080231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725755+0000","last_change":"2024-09-21T02:09:11.561183+0000","last_active":"2024-09-21T02:09:16.725755+0000","last_peered":"2024-09-21T02:09:16.725755+0000","last_clean":"2024-09-21T02:09:16.725755+0000","last_became_active":"2024-09-21T02:09:11.560260+0000","last_became_peered":"2024-09-21T02:09:11.560260+0000","last_unstale":"2024-09-21T02:09:16.725755+0000","last_undegraded":"2024-09-21T02:09:16.725755+0000","last_fullsized":"2024-09-21T02:09:16.725755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.593415+0000","last_change":"2024-09-21T02:09:11.540272+0000","last_active":"2024-09-21T02:09:16.593415+0000","last_peered":"2024-09-21T02:09:16.593415+0000","last_clean":"2024-09-21T02:09:16.593415+0000","last_became_active":"2024-09-21T02:09:11.539409+0000","last_became_peered":"2024-09-21T02:09:11.539409+0000","last_unstale":"2024-09-21T02:09:16.593415+0000","last_undegraded":"2024-09-21T02:09:16.593415+0000","last_fullsized":"2024-09-21T02:09:16.593415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:28.820372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.721399+0000","last_change":"2024-09-21T02:09:11.544995+0000","last_active":"2024-09-21T02:09:16.721399+0000","last_peered":"2024-09-21T02:09:16.721399+0000","last_clean":"2024-09-21T02:09:16.721399+0000","last_became_active":"2024-09-21T02:09:11.544306+0000","last_became_peered":"2024-09-21T02:09:11.544306+0000","last_unstale":"2024-09-21T02:09:16.721399+0000","last_undegraded":"2024-09-21T02:09:16.721399+0000","last_fullsized":"2024-09-21T02:09:16.721399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:55.026924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725507+0000","last_change":"2024-09-21T02:09:11.582210+0000","last_active":"2024-09-21T02:09:16.725507+0000","last_peered":"2024-09-21T02:09:16.725507+0000","last_clean":"2024-09-21T02:09:16.725507+0000","last_became_active":"2024-09-21T02:09:11.581734+0000","last_became_peered":"2024-09-21T02:09:11.581734+0000","last_unstale":"2024-09-21T02:09:16.725507+0000","last_undegraded":"2024-09-21T02:09:16.725507+0000","last_fullsized":"2024-09-21T02:09:16.725507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:01.926437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723576+0000","last_change":"2024-09-21T02:09:11.574300+0000","last_active":"2024-09-21T02:09:16.723576+0000","last_peered":"2024-09-21T02:09:16.723576+0000","last_clean":"2024-09-21T02:09:16.723576+0000","last_became_active":"2024-09-21T02:09:11.573443+0000","last_became_peered":"2024-09-21T02:09:11.573443+0000","last_unstale":"2024-09-21T02:09:16.723576+0000","last_undegraded":"2024-09-21T02:09:16.723576+0000","last_fullsized":"2024-09-21T02:09:16.723576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.720120+0000","last_change":"2024-09-21T02:09:11.518238+0000","last_active":"2024-09-21T02:09:16.720120+0000","last_peered":"2024-09-21T02:09:16.720120+0000","last_clean":"2024-09-21T02:09:16.720120+0000","last_became_active":"2024-09-21T02:09:11.517379+0000","last_became_peered":"2024-09-21T02:09:11.517379+0000","last_unstale":"2024-09-21T02:09:16.720120+0000","last_undegraded":"2024-09-21T02:09:16.720120+0000","last_fullsized":"2024-09-21T02:09:16.720120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:42.969908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723824+0000","last_change":"2024-09-21T02:09:11.536912+0000","last_active":"2024-09-21T02:09:16.723824+0000","last_peered":"2024-09-21T02:09:16.723824+0000","last_clean":"2024-09-21T02:09:16.723824+0000","last_became_active":"2024-09-21T02:09:11.535607+0000","last_became_peered":"2024-09-21T02:09:11.535607+0000","last_unstale":"2024-09-21T02:09:16.723824+0000","last_undegraded":"2024-09-21T02:09:16.723824+0000","last_fullsized":"2024-09-21T02:09:16.723824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:23.006299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.589439+0000","last_change":"2024-09-21T02:09:11.546912+0000","last_active":"2024-09-21T02:09:16.589439+0000","last_peered":"2024-09-21T02:09:16.589439+0000","last_clean":"2024-09-21T02:09:16.589439+0000","last_became_active":"2024-09-21T02:09:11.543438+0000","last_became_peered":"2024-09-21T02:09:11.543438+0000","last_unstale":"2024-09-21T02:09:16.589439+0000","last_undegraded":"2024-09-21T02:09:16.589439+0000","last_fullsized":"2024-09-21T02:09:16.589439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:17.962352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.595165+0000","last_change":"2024-09-21T02:09:11.547068+0000","last_active":"2024-09-21T02:09:16.595165+0000","last_peered":"2024-09-21T02:09:16.595165+0000","last_clean":"2024-09-21T02:09:16.595165+0000","last_became_active":"2024-09-21T02:09:11.545661+0000","last_became_peered":"2024-09-21T02:09:11.545661+0000","last_unstale":"2024-09-21T02:09:16.595165+0000","last_undegraded":"2024-09-21T02:09:16.595165+0000","last_fullsized":"2024-09-21T02:09:16.595165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:00.421056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.725978+0000","last_change":"2024-09-21T02:09:11.571193+0000","last_active":"2024-09-21T02:09:16.725978+0000","last_peered":"2024-09-21T02:09:16.725978+0000","last_clean":"2024-09-21T02:09:16.725978+0000","last_became_active":"2024-09-21T02:09:11.563913+0000","last_became_peered":"2024-09-21T02:09:11.563913+0000","last_unstale":"2024-09-21T02:09:16.725978+0000","last_undegraded":"2024-09-21T02:09:16.725978+0000","last_fullsized":"2024-09-21T02:09:16.725978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.726661+0000","last_change":"2024-09-21T02:09:11.564018+0000","last_active":"2024-09-21T02:09:16.726661+0000","last_peered":"2024-09-21T02:09:16.726661+0000","last_clean":"2024-09-21T02:09:16.726661+0000","last_became_active":"2024-09-21T02:09:11.563374+0000","last_became_peered":"2024-09-21T02:09:11.563374+0000","last_unstale":"2024-09-21T02:09:16.726661+0000","last_undegraded":"2024-09-21T02:09:16.726661+0000","last_fullsized":"2024-09-21T02:09:16.726661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592542+0000","last_change":"2024-09-21T02:09:11.571561+0000","last_active":"2024-09-21T02:09:16.592542+0000","last_peered":"2024-09-21T02:09:16.592542+0000","last_clean":"2024-09-21T02:09:16.592542+0000","last_became_active":"2024-09-21T02:09:11.570942+0000","last_became_peered":"2024-09-21T02:09:11.570942+0000","last_unstale":"2024-09-21T02:09:16.592542+0000","last_undegraded":"2024-09-21T02:09:16.592542+0000","last_fullsized":"2024-09-21T02:09:16.592542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:07.001076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.588463+0000","last_change":"2024-09-21T02:09:11.568051+0000","last_active":"2024-09-21T02:09:16.588463+0000","last_peered":"2024-09-21T02:09:16.588463+0000","last_clean":"2024-09-21T02:09:16.588463+0000","last_became_active":"2024-09-21T02:09:11.567478+0000","last_became_peered":"2024-09-21T02:09:11.567478+0000","last_unstale":"2024-09-21T02:09:16.588463+0000","last_undegraded":"2024-09-21T02:09:16.588463+0000","last_fullsized":"2024-09-21T02:09:16.588463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:40.465900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591178+0000","last_change":"2024-09-21T02:09:11.536755+0000","last_active":"2024-09-21T02:09:16.591178+0000","last_peered":"2024-09-21T02:09:16.591178+0000","last_clean":"2024-09-21T02:09:16.591178+0000","last_became_active":"2024-09-21T02:09:11.535871+0000","last_became_peered":"2024-09-21T02:09:11.535871+0000","last_unstale":"2024-09-21T02:09:16.591178+0000","last_undegraded":"2024-09-21T02:09:16.591178+0000","last_fullsized":"2024-09-21T02:09:16.591178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:00.938047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.724464+0000","last_change":"2024-09-21T02:09:11.572153+0000","last_active":"2024-09-21T02:09:16.724464+0000","last_peered":"2024-09-21T02:09:16.724464+0000","last_clean":"2024-09-21T02:09:16.724464+0000","last_became_active":"2024-09-21T02:09:11.567470+0000","last_became_peered":"2024-09-21T02:09:11.567470+0000","last_unstale":"2024-09-21T02:09:16.724464+0000","last_undegraded":"2024-09-21T02:09:16.724464+0000","last_fullsized":"2024-09-21T02:09:16.724464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:24.775708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.591826+0000","last_change":"2024-09-21T02:09:11.532323+0000","last_active":"2024-09-21T02:09:16.591826+0000","last_peered":"2024-09-21T02:09:16.591826+0000","last_clean":"2024-09-21T02:09:16.591826+0000","last_became_active":"2024-09-21T02:09:11.531480+0000","last_became_peered":"2024-09-21T02:09:11.531480+0000","last_unstale":"2024-09-21T02:09:16.591826+0000","last_undegraded":"2024-09-21T02:09:16.591826+0000","last_fullsized":"2024-09-21T02:09:16.591826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:40.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.723696+0000","last_change":"2024-09-21T02:09:11.575204+0000","last_active":"2024-09-21T02:09:16.723696+0000","last_peered":"2024-09-21T02:09:16.723696+0000","last_clean":"2024-09-21T02:09:16.723696+0000","last_became_active":"2024-09-21T02:09:11.567379+0000","last_became_peered":"2024-09-21T02:09:11.567379+0000","last_unstale":"2024-09-21T02:09:16.723696+0000","last_undegraded":"2024-09-21T02:09:16.723696+0000","last_fullsized":"2024-09-21T02:09:16.723696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:48.813808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.722559+0000","last_change":"2024-09-21T02:09:11.540599+0000","last_active":"2024-09-21T02:09:16.722559+0000","last_peered":"2024-09-21T02:09:16.722559+0000","last_clean":"2024-09-21T02:09:16.722559+0000","last_became_active":"2024-09-21T02:09:11.539829+0000","last_became_peered":"2024-09-21T02:09:11.539829+0000","last_unstale":"2024-09-21T02:09:16.722559+0000","last_undegraded":"2024-09-21T02:09:16.722559+0000","last_fullsized":"2024-09-21T02:09:16.722559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:54.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.719555+0000","last_change":"2024-09-21T02:09:11.534649+0000","last_active":"2024-09-21T02:09:16.719555+0000","last_peered":"2024-09-21T02:09:16.719555+0000","last_clean":"2024-09-21T02:09:16.719555+0000","last_became_active":"2024-09-21T02:09:11.533392+0000","last_became_peered":"2024-09-21T02:09:11.533392+0000","last_unstale":"2024-09-21T02:09:16.719555+0000","last_undegraded":"2024-09-21T02:09:16.719555+0000","last_fullsized":"2024-09-21T02:09:16.719555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.169470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T02:09:16.592021+0000","last_change":"2024-09-21T02:09:11.546630+0000","last_active":"2024-09-21T02:09:16.592021+0000","last_peered":"2024-09-21T02:09:16.592021+0000","last_clean":"2024-09-21T02:09:16.592021+0000","last_became_active":"2024-09-21T02:09:11.545498+0000","last_became_peered":"2024-09-21T02:09:11.545498+0000","last_unstale":"2024-09-21T02:09:16.592021+0000","last_undegraded":"2024-09-21T02:09:16.592021+0000","last_fullsized":"2024-09-21T02:09:16.592021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:10.481722+0000","last_clean_scrub_stamp":"2024-09-21T02:09:10.481722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:25.840553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":699032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":108172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":111924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":108082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":282624,"data_stored":108242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":699088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:09:39.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:09:39.047 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T02:09:39.047 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T02:09:39.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T02:09:39.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 -- 172.21.15.96:0/1344274738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698155240 msgr2=0x7fd698073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 --2- 172.21.15.96:0/1344274738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698155240 0x7fd698073430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd688009c80 tx=0x7fd68801cb90 comp rx=0 tx=0).stop 2024-09-21T02:09:39.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 -- 172.21.15.96:0/1344274738 shutdown_connections 2024-09-21T02:09:39.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 --2- 172.21.15.96:0/1344274738 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698073970 0x7fd69806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 --2- 172.21.15.96:0/1344274738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698155240 0x7fd698073430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 -- 172.21.15.96:0/1344274738 >> 172.21.15.96:0/1344274738 conn(0x7fd69806eea0 msgr2=0x7fd69806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 -- 172.21.15.96:0/1344274738 shutdown_connections 2024-09-21T02:09:39.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.228+0000 7fd69d42a640 1 -- 172.21.15.96:0/1344274738 wait complete. 2024-09-21T02:09:39.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 Processor -- start 2024-09-21T02:09:39.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 -- start start 2024-09-21T02:09:39.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd69806e3e0 0x7fd698164400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 0x7fd6981523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698155240 0x7fd6980b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd698168720 con 0x7fd698073970 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6981685a0 con 0x7fd69806e3e0 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd6981688a0 con 0x7fd698155240 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 0x7fd6981523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 0x7fd6981523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42724/0 (socket says 172.21.15.96:42724) 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 -- 172.21.15.96:0/4180419169 learned_addr learned my addr 172.21.15.96:0/4180419169 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd696ffd640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd69806e3e0 0x7fd698164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6977fe640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698155240 0x7fd6980b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 -- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698155240 msgr2=0x7fd6980b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698155240 0x7fd6980b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 -- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd69806e3e0 msgr2=0x7fd698164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd69806e3e0 0x7fd698164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 -- 172.21.15.96:0/4180419169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd698069570 con 0x7fd698073970 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd696ffd640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd69806e3e0 0x7fd698164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6977fe640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698155240 0x7fd6980b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd6967fc640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 0x7fd6981523b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00b410 tx=0x7fd68c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c015020 con 0x7fd698073970 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd68c011920 con 0x7fd698073970 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd698155e20 con 0x7fd698073970 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c011d70 con 0x7fd698073970 2024-09-21T02:09:39.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.232+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6981563e0 con 0x7fd698073970 2024-09-21T02:09:39.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.236+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd69806dea0 con 0x7fd698073970 2024-09-21T02:09:39.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.236+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd68c018020 con 0x7fd698073970 2024-09-21T02:09:39.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.236+0000 7fd67f7fe640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd684073200 0x7fd684075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.236+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd68c0944e0 con 0x7fd698073970 2024-09-21T02:09:39.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.236+0000 7fd696ffd640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd684073200 0x7fd684075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.236+0000 7fd696ffd640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd684073200 0x7fd684075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd6880af000 tx=0x7fd68801b3d0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.240+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd68c01f020 con 0x7fd698073970 2024-09-21T02:09:39.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.432+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd69806e9d0 con 0x7fd698073970 2024-09-21T02:09:39.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.432+0000 7fd67f7fe640 1 -- 172.21.15.96:0/4180419169 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd68c05f1c0 con 0x7fd698073970 2024-09-21T02:09:39.437 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:39.437 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T02:09:39.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd684073200 msgr2=0x7fd684075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd684073200 0x7fd684075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd6880af000 tx=0x7fd68801b3d0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 msgr2=0x7fd6981523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 0x7fd6981523b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00b410 tx=0x7fd68c00b8d0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 shutdown_connections 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd684073200 0x7fd684075710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd698155240 0x7fd6980b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd698073970 0x7fd6981523b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 --2- 172.21.15.96:0/4180419169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd69806e3e0 0x7fd698164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 >> 172.21.15.96:0/4180419169 conn(0x7fd69806eea0 msgr2=0x7fd69815b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 shutdown_connections 2024-09-21T02:09:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.436+0000 7fd69d42a640 1 -- 172.21.15.96:0/4180419169 wait complete. 2024-09-21T02:09:39.458 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T02:09:39.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:39.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 -- 172.21.15.96:0/2350172604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0cac80 msgr2=0x7f7dcc0cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/2350172604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0cac80 0x7f7dcc0cb090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009840 tx=0x7f7dbc01c790 comp rx=0 tx=0).stop 2024-09-21T02:09:39.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 -- 172.21.15.96:0/2350172604 shutdown_connections 2024-09-21T02:09:39.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/2350172604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0cac80 0x7f7dcc0cb090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 -- 172.21.15.96:0/2350172604 >> 172.21.15.96:0/2350172604 conn(0x7f7dcc06eea0 msgr2=0x7f7dcc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 -- 172.21.15.96:0/2350172604 shutdown_connections 2024-09-21T02:09:39.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 -- 172.21.15.96:0/2350172604 wait complete. 2024-09-21T02:09:39.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 Processor -- start 2024-09-21T02:09:39.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.604+0000 7f7dd40aa640 1 -- start start 2024-09-21T02:09:39.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dcc06dea0 0x7f7dcc1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dcc0cac80 0x7f7dcc152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 0x7f7dcc159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc16d110 con 0x7f7dcc0ce470 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc16cf90 con 0x7f7dcc06dea0 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc16d290 con 0x7f7dcc0cac80 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 0x7f7dcc159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 0x7f7dcc159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42740/0 (socket says 172.21.15.96:42740) 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 -- 172.21.15.96:0/794833049 learned_addr learned my addr 172.21.15.96:0/794833049 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd1645640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dcc0cac80 0x7f7dcc152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd1e46640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dcc06dea0 0x7f7dcc1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 -- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dcc0cac80 msgr2=0x7f7dcc152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dcc0cac80 0x7f7dcc152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 -- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dcc06dea0 msgr2=0x7f7dcc1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dcc06dea0 0x7f7dcc1595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 -- 172.21.15.96:0/794833049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc0c8f80 con 0x7f7dcc0ce470 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd2647640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 0x7f7dcc159f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c730 tx=0x7f7dc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4003b80 con 0x7f7dcc0ce470 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dc4003d40 con 0x7f7dcc0ce470 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc0c94b0 con 0x7f7dcc0ce470 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40052d0 con 0x7f7dcc0ce470 2024-09-21T02:09:39.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc0c9a70 con 0x7f7dcc0ce470 2024-09-21T02:09:39.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc0c9220 con 0x7f7dcc0ce470 2024-09-21T02:09:39.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.608+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7dc4005680 con 0x7f7dcc0ce470 2024-09-21T02:09:39.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.612+0000 7f7dc27fc640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7db0073200 0x7f7db0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.612+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7dc4094550 con 0x7f7dcc0ce470 2024-09-21T02:09:39.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.612+0000 7f7dd1e46640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7db0073200 0x7f7db0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.612+0000 7f7dd1e46640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7db0073200 0x7f7db0075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b810 tx=0x7f7dc8008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.616+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7dc405f230 con 0x7f7dcc0ce470 2024-09-21T02:09:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dcc0cb090 con 0x7f7dcc0ce470 2024-09-21T02:09:39.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dc27fc640 1 -- 172.21.15.96:0/794833049 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f7dc405eb30 con 0x7f7dcc0ce470 2024-09-21T02:09:39.821 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:39.821 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:09:39.821 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:09:39.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7db0073200 msgr2=0x7f7db0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7db0073200 0x7f7db0075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b810 tx=0x7f7dc8008040 comp rx=0 tx=0).stop 2024-09-21T02:09:39.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 msgr2=0x7f7dcc159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 0x7f7dcc159f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c730 tx=0x7f7dc400cbf0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 shutdown_connections 2024-09-21T02:09:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7db0073200 0x7f7db0075710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc0ce470 0x7f7dcc159f60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dcc0cac80 0x7f7dcc152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 --2- 172.21.15.96:0/794833049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dcc06dea0 0x7f7dcc1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 >> 172.21.15.96:0/794833049 conn(0x7f7dcc06eea0 msgr2=0x7f7dcc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 shutdown_connections 2024-09-21T02:09:39.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.816+0000 7f7dd40aa640 1 -- 172.21.15.96:0/794833049 wait complete. 2024-09-21T02:09:39.841 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 -- 172.21.15.96:0/1521098516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 msgr2=0x7f775806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 --2- 172.21.15.96:0/1521098516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775806dea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7754009840 tx=0x7f775401c820 comp rx=0 tx=0).stop 2024-09-21T02:09:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 -- 172.21.15.96:0/1521098516 shutdown_connections 2024-09-21T02:09:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 --2- 172.21.15.96:0/1521098516 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 0x7f77580cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 --2- 172.21.15.96:0/1521098516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775806dea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 -- 172.21.15.96:0/1521098516 >> 172.21.15.96:0/1521098516 conn(0x7f775806eea0 msgr2=0x7f775806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 -- 172.21.15.96:0/1521098516 shutdown_connections 2024-09-21T02:09:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 -- 172.21.15.96:0/1521098516 wait complete. 2024-09-21T02:09:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 Processor -- start 2024-09-21T02:09:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.976+0000 7f775ffe5640 1 -- start start 2024-09-21T02:09:39.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775806e3e0 0x7f7758158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 0x7f7758158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7758168770 con 0x7f77580ced00 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77581685f0 con 0x7f775806e3e0 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f77581688f0 con 0x7f775815c5b0 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775814fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42758/0 (socket says 172.21.15.96:42758) 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 -- 172.21.15.96:0/731498800 learned_addr learned my addr 172.21.15.96:0/731498800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775e582640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 0x7f7758158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775dd81640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775806e3e0 0x7f7758158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 -- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 msgr2=0x7f7758158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 0x7f7758158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 -- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775806e3e0 msgr2=0x7f7758158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775806e3e0 0x7f7758158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 -- 172.21.15.96:0/731498800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77581522b0 con 0x7f77580ced00 2024-09-21T02:09:39.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775e582640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 0x7f7758158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:39.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775d580640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775814fb90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f774800c3c0 tx=0x7f774800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748017070 con 0x7f77580ced00 2024-09-21T02:09:39.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7748003600 con 0x7f77580ced00 2024-09-21T02:09:39.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748003a50 con 0x7f77580ced00 2024-09-21T02:09:39.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7758152590 con 0x7f77580ced00 2024-09-21T02:09:39.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77580b55c0 con 0x7f77580ced00 2024-09-21T02:09:39.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775806dea0 con 0x7f77580ced00 2024-09-21T02:09:39.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f774801a070 con 0x7f77580ced00 2024-09-21T02:09:39.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f774e7fc640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f773c073200 0x7f773c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7748093700 con 0x7f77580ced00 2024-09-21T02:09:39.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.980+0000 7f775dd81640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f773c073200 0x7f773c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.984+0000 7f775dd81640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f773c073200 0x7f773c075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77540af000 tx=0x7f775401b060 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:39.984+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f775806e7f0 con 0x7f77580ced00 2024-09-21T02:09:40.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.184+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f775806f500 con 0x7f77580ced00 2024-09-21T02:09:40.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.184+0000 7f774e7fc640 1 -- 172.21.15.96:0/731498800 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f774805e500 con 0x7f77580ced00 2024-09-21T02:09:40.189 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:40.189 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:09:40.190 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f773c073200 msgr2=0x7f773c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f773c073200 0x7f773c075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77540af000 tx=0x7f775401b060 comp rx=0 tx=0).stop 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 msgr2=0x7f775814fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775814fb90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f774800c3c0 tx=0x7f774800c880 comp rx=0 tx=0).stop 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 shutdown_connections 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f773c073200 0x7f773c075710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f775815c5b0 0x7f7758158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77580ced00 0x7f775814fb90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 --2- 172.21.15.96:0/731498800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775806e3e0 0x7f7758158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 >> 172.21.15.96:0/731498800 conn(0x7f775806eea0 msgr2=0x7f775806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 shutdown_connections 2024-09-21T02:09:40.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.188+0000 7f775ffe5640 1 -- 172.21.15.96:0/731498800 wait complete. 2024-09-21T02:09:40.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:09:40.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3010240509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 msgr2=0x7faddc003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3010240509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7faddc003eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fade4001720 tx=0x7fade401c7b0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf3fff640 1 -- 172.21.15.96:0/3010240509 reap_dead start 2024-09-21T02:09:40.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3010240509 shutdown_connections 2024-09-21T02:09:40.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3010240509 >> 172.21.15.96:0/3010240509 conn(0x7fadf406eea0 msgr2=0x7fadf406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3010240509 shutdown_connections 2024-09-21T02:09:40.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.344+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3010240509 wait complete. 2024-09-21T02:09:40.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 Processor -- start 2024-09-21T02:09:40.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- start start 2024-09-21T02:09:40.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7fadf414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadf414e440 0x7fadf41fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fadf41fa760 0x7fadf4160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4168cf0 con 0x7faddc003a40 2024-09-21T02:09:40.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadf415cb90 con 0x7fadf414e440 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fadf4168e70 con 0x7fadf41fa760 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7fadf414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7fadf414df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42774/0 (socket says 172.21.15.96:42774) 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 -- 172.21.15.96:0/3995603023 learned_addr learned my addr 172.21.15.96:0/3995603023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 -- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fadf41fa760 msgr2=0x7fadf4160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fadf41fa760 0x7fadf4160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 -- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadf414e440 msgr2=0x7fadf41fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadf414e440 0x7fadf41fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 -- 172.21.15.96:0/3995603023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf41603b0 con 0x7faddc003a40 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf37fe640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7fadf414df00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fade4000c10 tx=0x7fade4003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade40bd070 con 0x7faddc003a40 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fade4006500 con 0x7faddc003a40 2024-09-21T02:09:40.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf40b5a70 con 0x7faddc003a40 2024-09-21T02:09:40.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4006930 con 0x7faddc003a40 2024-09-21T02:09:40.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf40b54b0 con 0x7faddc003a40 2024-09-21T02:09:40.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.348+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf40c4580 con 0x7faddc003a40 2024-09-21T02:09:40.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.352+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fade40c0070 con 0x7faddc003a40 2024-09-21T02:09:40.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.352+0000 7faddbfff640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fade0073200 0x7fade0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.352+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fade412e200 con 0x7faddc003a40 2024-09-21T02:09:40.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.352+0000 7fadf2ffd640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fade0073200 0x7fade0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.352+0000 7fadf2ffd640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fade0073200 0x7fade0075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fade80058d0 tx=0x7fade8005d50 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.356+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fade40aa050 con 0x7faddc003a40 2024-09-21T02:09:40.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.528+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadf4073310 con 0x7faddc003a40 2024-09-21T02:09:40.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.528+0000 7faddbfff640 1 -- 172.21.15.96:0/3995603023 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18598 (secure 0 0 0) 0x7fade4016510 con 0x7faddc003a40 2024-09-21T02:09:40.533 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:40.533 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:15.990483+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:09:40.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fade0073200 msgr2=0x7fade0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fade0073200 0x7fade0075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fade80058d0 tx=0x7fade8005d50 comp rx=0 tx=0).stop 2024-09-21T02:09:40.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 msgr2=0x7fadf414df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7fadf414df00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fade4000c10 tx=0x7fade4003be0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 shutdown_connections 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fade0073200 0x7fade0075710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fadf41fa760 0x7fadf4160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadf414e440 0x7fadf41fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 --2- 172.21.15.96:0/3995603023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc003a40 0x7fadf414df00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 >> 172.21.15.96:0/3995603023 conn(0x7fadf406eea0 msgr2=0x7fadf41652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 shutdown_connections 2024-09-21T02:09:40.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.532+0000 7fadf9e1b640 1 -- 172.21.15.96:0/3995603023 wait complete. 2024-09-21T02:09:40.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:09:40.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 -- 172.21.15.96:0/1715455808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6f8003a10 msgr2=0x7fe6f8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 --2- 172.21.15.96:0/1715455808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6f8003a10 0x7fe6f8003e80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe704009ee0 tx=0x7fe70401d210 comp rx=0 tx=0).stop 2024-09-21T02:09:40.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe711e31640 1 -- 172.21.15.96:0/1715455808 reap_dead start 2024-09-21T02:09:40.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 -- 172.21.15.96:0/1715455808 shutdown_connections 2024-09-21T02:09:40.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 -- 172.21.15.96:0/1715455808 >> 172.21.15.96:0/1715455808 conn(0x7fe70c06eea0 msgr2=0x7fe70c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 -- 172.21.15.96:0/1715455808 shutdown_connections 2024-09-21T02:09:40.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 -- 172.21.15.96:0/1715455808 wait complete. 2024-09-21T02:09:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 Processor -- start 2024-09-21T02:09:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.704+0000 7fe713894640 1 -- start start 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8003a10 0x7fe70c14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe70c14c7e0 0x7fe70c14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 0x7fe70c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c0cb470 con 0x7fe70c14bf90 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c0cb2f0 con 0x7fe6f8003a10 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe70c0cb5f0 con 0x7fe70c14c7e0 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 0x7fe70c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 0x7fe70c15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42788/0 (socket says 172.21.15.96:42788) 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 -- 172.21.15.96:0/485210624 learned_addr learned my addr 172.21.15.96:0/485210624 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe710e2f640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe70c14c7e0 0x7fe70c14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711630640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8003a10 0x7fe70c14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 -- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe70c14c7e0 msgr2=0x7fe70c14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe70c14c7e0 0x7fe70c14e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 -- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8003a10 msgr2=0x7fe70c14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8003a10 0x7fe70c14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 -- 172.21.15.96:0/485210624 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c14d6f0 con 0x7fe70c14bf90 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe710e2f640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe70c14c7e0 0x7fe70c14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711630640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8003a10 0x7fe70c14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe711e31640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 0x7fe70c15c380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe70c071a50 tx=0x7fe6fc00e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc005020 con 0x7fe70c14bf90 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6fc0052d0 con 0x7fe70c14bf90 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c14d9d0 con 0x7fe70c14bf90 2024-09-21T02:09:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0047b0 con 0x7fe70c14bf90 2024-09-21T02:09:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c14df90 con 0x7fe70c14bf90 2024-09-21T02:09:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c14d460 con 0x7fe70c14bf90 2024-09-21T02:09:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6fc01a020 con 0x7fe70c14bf90 2024-09-21T02:09:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.708+0000 7fe701ffb640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe6f0073200 0x7fe6f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.712+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe6fc094e50 con 0x7fe70c14bf90 2024-09-21T02:09:40.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.712+0000 7fe711630640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe6f0073200 0x7fe6f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.712+0000 7fe711630640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe6f0073200 0x7fe6f0075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ae000 tx=0x7fe704019fb0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.712+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe6fc05fc10 con 0x7fe70c14bf90 2024-09-21T02:09:40.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.916+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe70c073430 con 0x7fe70c14bf90 2024-09-21T02:09:40.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.916+0000 7fe701ffb640 1 -- 172.21.15.96:0/485210624 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7fe6fc005720 con 0x7fe70c14bf90 2024-09-21T02:09:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:09:40.923 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:09:40.923 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:09:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe6f0073200 msgr2=0x7fe6f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe6f0073200 0x7fe6f0075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ae000 tx=0x7fe704019fb0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 msgr2=0x7fe70c15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 0x7fe70c15c380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe70c071a50 tx=0x7fe6fc00e9b0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 shutdown_connections 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe6f0073200 0x7fe6f0075710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c14bf90 0x7fe70c15c380 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe70c14c7e0 0x7fe70c14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 --2- 172.21.15.96:0/485210624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8003a10 0x7fe70c14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 >> 172.21.15.96:0/485210624 conn(0x7fe70c06eea0 msgr2=0x7fe70c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 shutdown_connections 2024-09-21T02:09:40.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:40.920+0000 7fe713894640 1 -- 172.21.15.96:0/485210624 wait complete. 2024-09-21T02:09:41.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:09:10.893814+0000', 'modified': '2024-09-21T02:09:31.976954+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4353}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4353': {'gid': 4353, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/2262930956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2262930956}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2262930956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4353, 'qdb_cluster': [4353]} 2024-09-21T02:09:41.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:09:41.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:09:41.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 -- 172.21.15.96:0/2023175987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815f750 msgr2=0x7f81080cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 --2- 172.21.15.96:0/2023175987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815f750 0x7f81080cff20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009840 tx=0x7f80f801c7f0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 -- 172.21.15.96:0/2023175987 shutdown_connections 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 --2- 172.21.15.96:0/2023175987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815f750 0x7f81080cff20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 --2- 172.21.15.96:0/2023175987 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f810815b310 0x7f81080cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 -- 172.21.15.96:0/2023175987 >> 172.21.15.96:0/2023175987 conn(0x7f810806fbe0 msgr2=0x7f810806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 -- 172.21.15.96:0/2023175987 shutdown_connections 2024-09-21T02:09:41.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.500+0000 7f81102eb640 1 -- 172.21.15.96:0/2023175987 wait complete. 2024-09-21T02:09:41.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 Processor -- start 2024-09-21T02:09:41.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 -- start start 2024-09-21T02:09:41.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 0x7f810814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f810815f750 0x7f81081fab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f810815fb60 0x7f8108156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8108168f10 con 0x7f810815b310 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8108168d90 con 0x7f810815fb60 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8108169090 con 0x7f810815f750 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 0x7f810814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 0x7f810814d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42798/0 (socket says 172.21.15.96:42798) 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 -- 172.21.15.96:0/435361104 learned_addr learned my addr 172.21.15.96:0/435361104 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e888640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f810815fb60 0x7f8108156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810d886640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f810815f750 0x7f81081fab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 -- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f810815f750 msgr2=0x7f81081fab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f810815f750 0x7f81081fab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 -- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f810815fb60 msgr2=0x7f8108156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f810815fb60 0x7f8108156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 -- 172.21.15.96:0/435361104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81081fb050 con 0x7f810815b310 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e888640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f810815fb60 0x7f8108156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f810e087640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 0x7f810814d0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f810400d950 tx=0x7f810400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f80feffd640 1 -- 172.21.15.96:0/435361104 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104017070 con 0x7f810815b310 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f80feffd640 1 -- 172.21.15.96:0/435361104 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8104009d20 con 0x7f810815b310 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f80feffd640 1 -- 172.21.15.96:0/435361104 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040106c0 con 0x7f810815b310 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8108151640 con 0x7f810815b310 2024-09-21T02:09:41.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.504+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8108153030 con 0x7f810815b310 2024-09-21T02:09:41.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f80feffd640 1 -- 172.21.15.96:0/435361104 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f810401a070 con 0x7f810815b310 2024-09-21T02:09:41.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f80feffd640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f80ec073230 0x7f80ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f80feffd640 1 -- 172.21.15.96:0/435361104 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f81040945f0 con 0x7f810815b310 2024-09-21T02:09:41.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f80ec073230 0x7f80ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 Processor -- start 2024-09-21T02:09:41.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 -- start start 2024-09-21T02:09:41.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f80ec073230 0x7f80ec075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f80f80af000 tx=0x7f80f801b030 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 0x56492e2e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e17cca0 0x56492e2e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56492e1b9410 con 0x56492e193950 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56492e0f6b40 con 0x56492dda3460 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x56492e0e5f70 con 0x56492e17cca0 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 0x56492e2e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 0x56492e2e4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42814/0 (socket says 172.21.15.96:42814) 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 -- 172.21.15.96:0/3154618427 learned_addr learned my addr 172.21.15.96:0/3154618427 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810e087640 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810e888640 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e17cca0 0x56492e2e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 -- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e17cca0 msgr2=0x56492e2e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e17cca0 0x56492e2e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 -- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 msgr2=0x56492e2e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 -- 172.21.15.96:0/3154618427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56492e0ad5b0 con 0x56492e193950 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.508+0000 7f810d886640 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 0x56492e2e4be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005ec0 tx=0x7f80f80b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f510d73677eb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f80ebfff640 1 -- 172.21.15.96:0/3154618427 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80c8070 con 0x56492e193950 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f80ebfff640 1 -- 172.21.15.96:0/3154618427 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f80042c0 con 0x56492e193950 2024-09-21T02:09:41.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f80ebfff640 1 -- 172.21.15.96:0/3154618427 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80b2040 con 0x56492e193950 2024-09-21T02:09:41.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- 172.21.15.96:0/3154618427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 msgr2=0x56492e2e4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 0x56492e2e4be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005ec0 tx=0x7f80f80b4000 comp rx=0 tx=0).stop 2024-09-21T02:09:41.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- 172.21.15.96:0/3154618427 shutdown_connections 2024-09-21T02:09:41.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e17cca0 0x56492e2e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e193950 0x56492e2e4be0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- 172.21.15.96:0/3154618427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- 172.21.15.96:0/3154618427 >> 172.21.15.96:0/3154618427 conn(0x56492e269d90 msgr2=0x56492e299690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:41.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- 172.21.15.96:0/3154618427 shutdown_connections 2024-09-21T02:09:41.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- 172.21.15.96:0/3154618427 wait complete. 2024-09-21T02:09:41.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- start start 2024-09-21T02:09:41.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 Processor -- start 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e2ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e2ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56492e0805f0 con 0x56492e17cca0 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56492e0e9d80 con 0x56492dda3460 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x56492e08c520 con 0x56492e193950 2024-09-21T02:09:41.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 10 client.0 _ll_register_callbacks cb 0x56492e090830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e2ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e2ebe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42828/0 (socket says 172.21.15.96:42828) 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 -- 172.21.15.96:0/3487653817 learned_addr learned my addr 172.21.15.96:0/3487653817 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810e888640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e2ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810e087640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 -- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 msgr2=0x56492e2ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e2ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 -- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 msgr2=0x56492e2eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2eb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 -- 172.21.15.96:0/3487653817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56492e084190 con 0x56492e17cca0 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810e888640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e2ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:41.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f810d886640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e2ebe60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80f801a420 tx=0x7f80f8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f065285143bdff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f80e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f80e97fa640 1 -- 172.21.15.96:0/3487653817 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80c8070 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f80e97fa640 1 -- 172.21.15.96:0/3487653817 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f80f80cddf0 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.512+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56492e07e110 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 1 -- 172.21.15.96:0/3487653817 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f80f80b2040 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 10 client.4770 fetch_fsmap learned FSMap version 9 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56492e16af10 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 1 -- 172.21.15.96:0/3487653817 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56492e07e110 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 1 -- 172.21.15.96:0/3487653817 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f80f80b5030 con 0x56492e17cca0 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 10 client.4770 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 10 client.4770 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 20 client.4770 populate_metadata read hostname 'smithi096' 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd887d0 0x56492e2f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 4 client.4770 mds_command: new command op to 4353 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 --> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56492e0f8a90 con 0x56492dd887d0 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f810e087640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd887d0 0x56492e2f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 10 client.4770 ms_handle_connect on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f810e087640 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd887d0 0x56492e2f5d00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75826dc535e18c5f server_cookie=df03c07c6298cebc in_seq=0 out_seq=0 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 1 -- 172.21.15.96:0/3487653817 <== mds.0 v2:172.21.15.96:6834/2262930956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56492e0f8a90 con 0x56492dd887d0 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 10 client.4770 handle_command_reply: tid=0 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 client.4770 shutdown 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 msgr2=0x56492e2ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e2ebe60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80f801a420 tx=0x7f80f8009500 comp rx=0 tx=0).stop 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 shutdown_connections 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd887d0 0x56492e2f5d00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e2ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e2ebe60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f80e97fa640 0 client.4770 ms_handle_reset on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 --2- 172.21.15.96:0/3487653817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e2eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 >> 172.21.15.96:0/3487653817 conn(0x56492e269d90 msgr2=0x56492e2a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 shutdown_connections 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 1 -- 172.21.15.96:0/3487653817 wait complete. 2024-09-21T02:09:41.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.516+0000 7f81175b6000 20 client.4770 trim_cache size 0 max 16384 2024-09-21T02:09:41.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.532+0000 7f81175b6000 1 Processor -- start 2024-09-21T02:09:41.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- start start 2024-09-21T02:09:41.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e35ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e35b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8108204a90 con 0x56492e17cca0 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8108203a90 con 0x56492dda3460 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8108202df0 con 0x56492e193950 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e35b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e35b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42840/0 (socket says 172.21.15.96:42840) 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 -- 172.21.15.96:0/1166017547 learned_addr learned my addr 172.21.15.96:0/1166017547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:41.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 -- 172.21.15.96:0/1166017547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 msgr2=0x56492e35b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 -- 172.21.15.96:0/1166017547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 msgr2=0x56492e35ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e35ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 -- 172.21.15.96:0/1166017547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56492e141db0 con 0x56492e17cca0 2024-09-21T02:09:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f810d886640 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e35b190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f80f8003620 tx=0x7f80f80a5ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a0366d462a7f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f80eb7fe640 1 -- 172.21.15.96:0/1166017547 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80c8040 con 0x56492e17cca0 2024-09-21T02:09:41.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f80eb7fe640 1 -- 172.21.15.96:0/1166017547 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f80036a0 con 0x56492e17cca0 2024-09-21T02:09:41.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f80eb7fe640 1 -- 172.21.15.96:0/1166017547 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8003ad0 con 0x56492e17cca0 2024-09-21T02:09:41.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- 172.21.15.96:0/1166017547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 msgr2=0x56492e35b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e35b190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f80f8003620 tx=0x7f80f80a5ca0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- 172.21.15.96:0/1166017547 shutdown_connections 2024-09-21T02:09:41.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492e17cca0 0x56492e35b190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 --2- 172.21.15.96:0/1166017547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492dda3460 0x56492e35ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- 172.21.15.96:0/1166017547 >> 172.21.15.96:0/1166017547 conn(0x56492e269d90 msgr2=0x56492e298cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:41.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- 172.21.15.96:0/1166017547 shutdown_connections 2024-09-21T02:09:41.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- 172.21.15.96:0/1166017547 wait complete. 2024-09-21T02:09:41.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 -- start start 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.536+0000 7f81175b6000 1 Processor -- start 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 0x56492e35e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492e17cca0 0x56492e35ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56492e0b1ee0 con 0x56492dda3460 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56492e0f5910 con 0x56492e17cca0 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x56492e0e9090 con 0x56492e193950 2024-09-21T02:09:41.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 10 client.0 _ll_register_callbacks cb 0x7f80f80c6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 0x56492e35e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 0x56492e35e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42848/0 (socket says 172.21.15.96:42848) 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 -- 172.21.15.96:0/3811325333 learned_addr learned my addr 172.21.15.96:0/3811325333 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e888640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810d886640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492e17cca0 0x56492e35ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 -- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 msgr2=0x56492e35f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 -- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492e17cca0 msgr2=0x56492e35ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492e17cca0 0x56492e35ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 -- 172.21.15.96:0/3811325333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56492e0acab0 con 0x56492dda3460 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e888640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810d886640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492e17cca0 0x56492e35ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810e087640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 0x56492e35e7e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f810405bed0 tx=0x7f8104009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcce739838df359 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:41.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 1 -- 172.21.15.96:0/3811325333 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104021040 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 1 -- 172.21.15.96:0/3811325333 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f81040842d0 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f80036a0 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 1 -- 172.21.15.96:0/3811325333 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f810405ec60 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 10 client.4776 fetch_fsmap learned FSMap version 9 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56492e08b070 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 1 -- 172.21.15.96:0/3811325333 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f80036a0 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 1 -- 172.21.15.96:0/3811325333 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f810405dec0 con 0x56492dda3460 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 10 client.4776 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 10 client.4776 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:09:41.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 20 client.4776 populate_metadata read hostname 'smithi096' 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd76830 0x56492e31b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 4 client.4776 mds_command: new command op to 4353 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 --> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] -- command(tid 0: {"prefix": "status"}) -- 0x56492e0f4540 con 0x56492dd76830 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810d886640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd76830 0x56492e31b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 10 client.4776 ms_handle_connect on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f810d886640 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd76830 0x56492e31b320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a46d47db773be4 server_cookie=e49a84a5268e8352 in_seq=0 out_seq=0 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 1 -- 172.21.15.96:0/3811325333 <== mds.0 v2:172.21.15.96:6834/2262930956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56492e0f4540 con 0x56492dd76830 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 10 client.4776 handle_command_reply: tid=0 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 client.4776 shutdown 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 msgr2=0x56492e35e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 0x56492e35e7e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f810405bed0 tx=0x7f8104009ef0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 shutdown_connections 2024-09-21T02:09:41.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:6834/2262930956,v1:172.21.15.96:6835/2262930956] conn(0x56492dd76830 0x56492e31b320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56492e193950 0x56492e35f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56492e17cca0 0x56492e35ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f80e97fa640 0 client.4776 ms_handle_reset on v2:172.21.15.96:6834/2262930956 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 --2- 172.21.15.96:0/3811325333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56492dda3460 0x56492e35e7e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 >> 172.21.15.96:0/3811325333 conn(0x56492e269d90 msgr2=0x56492e299690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 shutdown_connections 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 1 -- 172.21.15.96:0/3811325333 wait complete. 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81175b6000 20 client.4776 trim_cache size 0 max 16384 2024-09-21T02:09:41.558 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "892bf3fa-0a6f-4242-b0ca-536a4d478630", 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4353, 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 30.002757322000001, 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 9, 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 26, 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 26, 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 36.103922742000002 2024-09-21T02:09:41.559 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:09:41.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f80ec073230 msgr2=0x7f80ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f80ec073230 0x7f80ec075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f80f80af000 tx=0x7f80f801b030 comp rx=0 tx=0).stop 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 msgr2=0x7f810814d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 0x7f810814d0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f810400d950 tx=0x7f810400de10 comp rx=0 tx=0).stop 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 shutdown_connections 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f80ec073230 0x7f80ec075740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f810815fb60 0x7f8108156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f810815f750 0x7f81081fab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 --2- 172.21.15.96:0/435361104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f810815b310 0x7f810814d0b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 >> 172.21.15.96:0/435361104 conn(0x7f810806fbe0 msgr2=0x7f8108073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 shutdown_connections 2024-09-21T02:09:41.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:41.540+0000 7f81102eb640 1 -- 172.21.15.96:0/435361104 wait complete. 2024-09-21T02:09:41.564 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-21T02:09:41.574 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-21T02:09:41.574 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T02:09:41.574 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:41.575 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:41.575 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:41.575 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:09:41.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:41.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:41.576 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-21T02:09:41.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:41.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-21T02:09:41.642 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-09-21T02:09:41.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:09:41.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:41.644 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-21T02:09:41.650 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T02:09:41.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:41.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T02:09:41.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:41.708 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-09-21T02:09:41.889 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-21T02:09:41.889 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:09:41.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:09:42.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e4a056640 1 -- 172.21.15.96:0/2858923398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e30003a10 msgr2=0x7f2e30003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e4a056640 1 --2- 172.21.15.96:0/2858923398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e30003a10 0x7f2e30003e80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2e4408cc30 tx=0x7f2e3401cc40 comp rx=0 tx=0).stop 2024-09-21T02:09:42.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e43fff640 1 -- 172.21.15.96:0/2858923398 reap_dead start 2024-09-21T02:09:42.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e4a056640 1 -- 172.21.15.96:0/2858923398 shutdown_connections 2024-09-21T02:09:42.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e4a056640 1 -- 172.21.15.96:0/2858923398 >> 172.21.15.96:0/2858923398 conn(0x7f2e440d1b10 msgr2=0x7f2e440d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e4a056640 1 -- 172.21.15.96:0/2858923398 shutdown_connections 2024-09-21T02:09:42.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.040+0000 7f2e4a056640 1 -- 172.21.15.96:0/2858923398 wait complete. 2024-09-21T02:09:42.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 Processor -- start 2024-09-21T02:09:42.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 -- start start 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e30003a10 0x7f2e441fced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 0x7f2e44160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e44149d00 0x7f2e44160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e440d3380 con 0x7f2e441498f0 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4415fdf0 con 0x7f2e44149d00 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2e440d3500 con 0x7f2e30003a10 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 0x7f2e44160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 0x7f2e44160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42866/0 (socket says 172.21.15.96:42866) 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 -- 172.21.15.96:0/4268063538 learned_addr learned my addr 172.21.15.96:0/4268063538 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e437fe640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e30003a10 0x7f2e441fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e43fff640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e44149d00 0x7f2e44160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 -- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e30003a10 msgr2=0x7f2e441fced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e30003a10 0x7f2e441fced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 -- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e44149d00 msgr2=0x7f2e44160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e44149d00 0x7f2e44160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 -- 172.21.15.96:0/4268063538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441f9630 con 0x7f2e441498f0 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e3affd640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 0x7f2e44160a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c011a50 tx=0x7f2e2c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c01f070 con 0x7f2e441498f0 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e2c00fe80 con 0x7f2e441498f0 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441f9910 con 0x7f2e441498f0 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c012690 con 0x7f2e441498f0 2024-09-21T02:09:42.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.044+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e44154420 con 0x7f2e441498f0 2024-09-21T02:09:42.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.048+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e441f93a0 con 0x7f2e441498f0 2024-09-21T02:09:42.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.048+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2e2c022070 con 0x7f2e441498f0 2024-09-21T02:09:42.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.048+0000 7f2e40ff9640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2e24073200 0x7f2e24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.048+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e2c094fb0 con 0x7f2e441498f0 2024-09-21T02:09:42.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.048+0000 7f2e437fe640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2e24073200 0x7f2e24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.048+0000 7f2e437fe640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2e24073200 0x7f2e24075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e340af000 tx=0x7f2e340199e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.052+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2e2c05f630 con 0x7f2e441498f0 2024-09-21T02:09:42.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2e441f9e30 con 0x7f2e441498f0 2024-09-21T02:09:42.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e40ff9640 1 -- 172.21.15.96:0/4268063538 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f2e441f9e30 con 0x7f2e441498f0 2024-09-21T02:09:42.252 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2e24073200 msgr2=0x7f2e24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2e24073200 0x7f2e24075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e340af000 tx=0x7f2e340199e0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 msgr2=0x7f2e44160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 0x7f2e44160a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c011a50 tx=0x7f2e2c011f10 comp rx=0 tx=0).stop 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 shutdown_connections 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2e24073200 0x7f2e24075710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e44149d00 0x7f2e44160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e441498f0 0x7f2e44160a40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 --2- 172.21.15.96:0/4268063538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e30003a10 0x7f2e441fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 >> 172.21.15.96:0/4268063538 conn(0x7f2e440d1b10 msgr2=0x7f2e440b87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 shutdown_connections 2024-09-21T02:09:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42.248+0000 7f2e4a056640 1 -- 172.21.15.96:0/4268063538 wait complete. 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:09:42.273 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:09:42.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:42.274 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42471sec preferred_lft 42471sec 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:e1:97:6e:59 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:09:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:42.281 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:09:42.281 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:42.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:42.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:42.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:42.405 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:09:42.460 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-21T02:09:42.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:42.462 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-21T02:09:42.510 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-21T02:09:42.510 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:09:42.510 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:42.511 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:09:42.511 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:42.511 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:42.511 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:09:42.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:09:42.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:09:42.512 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:42.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:42.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:42.631 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-21T02:09:42.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:42.680 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-09-21T02:09:42.731 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:42.732 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:42.732 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:42.732 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T02:09:42.732 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:42.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:42.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:42.847 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T02:09:42.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T02:09:42.848 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:42.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:42.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:42.988 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:42.988 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:42.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-21T02:09:42.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:09:42.988 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:43.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:43.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:43.090 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:09:43.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:43.090 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:43.138 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:09:43.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:43.139 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:43.186 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:43.240 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:43.288 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:43.289 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:43.289 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:43.289 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:43.289 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:43.289 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:43.289 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:43.290 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:43.290 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:43.290 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:43.290 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:43.290 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:43.290 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:43.291 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:43.291 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:43.291 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:43.291 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:43.291 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:43.291 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:43.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:43.295 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:43.343 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:09:43.344 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:09:43.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:43.401 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:43.447 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:43.447 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:43.447 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:43.447 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:43.448 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:43.449 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:43.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:43.450 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:43.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-21T02:09:43.496+0000 7f1e53fdc580 -1 init, newargv = 0x7f1e40001f10 newargc=15 2024-09-21T02:09:43.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[36871]: starting ceph client 2024-09-21T02:09:43.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[36871]: starting fuse 2024-09-21T02:09:44.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:44.620 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:44.668 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:44.669 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:44.670 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:44.671 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:44.671 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:44.671 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:44.671 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:44.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:44.672 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:44.719 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-21T02:09:44.720 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-21T02:09:44.720 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:09:44.721 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:44.722 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-21T02:09:44.722 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-21T02:09:44.844 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36871.asok 2024-09-21T02:09:44.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:44.857 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36871.asok 2024-09-21T02:09:44.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:44.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36871.asok status 2024-09-21T02:09:45.038 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:09:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-21T02:09:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:09:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "36871", 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4791, 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-21T02:09:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4791 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2213560767 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:09:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2213560767 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4791 192.168.0.1:0/2213560767", 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/2213560767", 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 9, 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 26, 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-21T02:09:45.041 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:09:45.055 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:09:45.055 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.208+0000 7ff4a1462640 1 -- 172.21.15.96:0/3738344033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff488003a40 msgr2=0x7ff488003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.208+0000 7ff4a1462640 1 --2- 172.21.15.96:0/3738344033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff488003a40 0x7ff488003eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff484001dc0 tx=0x7ff48401cb30 comp rx=0 tx=0).stop 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.208+0000 7ff49b7fe640 1 -- 172.21.15.96:0/3738344033 reap_dead start 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- 172.21.15.96:0/3738344033 shutdown_connections 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- 172.21.15.96:0/3738344033 >> 172.21.15.96:0/3738344033 conn(0x7ff49c06eea0 msgr2=0x7ff49c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- 172.21.15.96:0/3738344033 shutdown_connections 2024-09-21T02:09:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- 172.21.15.96:0/3738344033 wait complete. 2024-09-21T02:09:45.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 Processor -- start 2024-09-21T02:09:45.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- start start 2024-09-21T02:09:45.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff49c0ca2d0 0x7ff49c1fc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff49c15d840 0x7ff49c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 0x7ff49c1f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c15c230 con 0x7ff49c159b20 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c15c0b0 con 0x7ff49c15d840 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff4a1462640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c0cc200 con 0x7ff49c0ca2d0 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 0x7ff49c1f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 0x7ff49c1f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42874/0 (socket says 172.21.15.96:42874) 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 -- 172.21.15.96:0/2093349257 learned_addr learned my addr 172.21.15.96:0/2093349257 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49a7fc640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff49c15d840 0x7ff49c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49affd640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff49c0ca2d0 0x7ff49c1fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 -- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff49c0ca2d0 msgr2=0x7ff49c1fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff49c0ca2d0 0x7ff49c1fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 -- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff49c15d840 msgr2=0x7ff49c1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff49c15d840 0x7ff49c1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49b7fe640 1 -- 172.21.15.96:0/2093349257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c1fc880 con 0x7ff49c159b20 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.212+0000 7ff49a7fc640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff49c15d840 0x7ff49c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff49b7fe640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 0x7ff49c1f8950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0cca70 tx=0x7ff49400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494015020 con 0x7ff49c159b20 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff494010980 con 0x7ff49c159b20 2024-09-21T02:09:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494010d90 con 0x7ff49c159b20 2024-09-21T02:09:45.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c0ca890 con 0x7ff49c159b20 2024-09-21T02:09:45.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c0c8390 con 0x7ff49c159b20 2024-09-21T02:09:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c075b80 con 0x7ff49c159b20 2024-09-21T02:09:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff494018020 con 0x7ff49c159b20 2024-09-21T02:09:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4937fe640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff47c073200 0x7ff47c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff494094850 con 0x7ff49c159b20 2024-09-21T02:09:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff49affd640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff47c073200 0x7ff47c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.216+0000 7ff49affd640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff47c073200 0x7ff47c075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4840aa000 tx=0x7ff484019920 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:45.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.220+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff49401f030 con 0x7ff49c159b20 2024-09-21T02:09:45.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.428+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff49c0b0d30 con 0x7ff49c159b20 2024-09-21T02:09:45.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.428+0000 7ff4937fe640 1 -- 172.21.15.96:0/2093349257 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff49c1fc880 con 0x7ff49c159b20 2024-09-21T02:09:45.434 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff47c073200 msgr2=0x7ff47c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff47c073200 0x7ff47c075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4840aa000 tx=0x7ff484019920 comp rx=0 tx=0).stop 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 msgr2=0x7ff49c1f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 0x7ff49c1f8950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0cca70 tx=0x7ff49400ea20 comp rx=0 tx=0).stop 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 shutdown_connections 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff47c073200 0x7ff47c075710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49c159b20 0x7ff49c1f8950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff49c15d840 0x7ff49c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 --2- 172.21.15.96:0/2093349257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff49c0ca2d0 0x7ff49c1fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 >> 172.21.15.96:0/2093349257 conn(0x7ff49c06eea0 msgr2=0x7ff49c15cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 shutdown_connections 2024-09-21T02:09:45.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45.432+0000 7ff4a1462640 1 -- 172.21.15.96:0/2093349257 wait complete. 2024-09-21T02:09:45.453 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:09:45.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:09:45.453 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-21T02:09:45.453 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T02:09:45.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:45.454 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:09:45.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:09:45.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:09:45.454 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T02:09:45.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.454 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42467sec preferred_lft 42467sec 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.461 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:e1:97:6e:59 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:09:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.463 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:09:45.513 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-21T02:09:45.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.514 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-21T02:09:45.562 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-21T02:09:45.562 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:09:45.562 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:09:45.562 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:45.563 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:09:45.563 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:45.563 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:45.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:09:45.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:09:45.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:09:45.564 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:45.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:45.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.686 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-21T02:09:45.731 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:09:45.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.732 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-09-21T02:09:45.779 INFO:teuthology.orchestra.run.smithi096.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T02:09:45.780 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:45.780 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:45.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:45.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T02:09:45.780 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:45.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.896 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T02:09:45.907 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:45.907 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether ea:bf:52:2f:f1:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.908 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::e8bf:52ff:fe2f:f11d/64 scope link 2024-09-21T02:09:45.909 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:45.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T02:09:45.960 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:45.960 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:45.960 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:45.960 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether ea:bf:52:2f:f1:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::e8bf:52ff:fe2f:f11d/64 scope link 2024-09-21T02:09:45.961 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:45.962 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T02:09:45.962 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:45.962 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:45.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T02:09:45.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T02:09:45.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T02:09:45.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T02:09:45.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T02:09:45.963 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:46.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:46.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:46.099 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:46.099 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:09:46.100 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-21T02:09:46.100 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T02:09:46.100 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:09:46.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:46.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:46.158 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-21T02:09:46.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:46.158 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:46.206 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T02:09:46.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:46.207 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:46.258 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:46.316 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:46.368 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:46.368 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:46.368 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:46.369 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:46.370 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:46.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:46.372 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:46.419 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-21T02:09:46.419 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-21T02:09:46.420 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-21T02:09:46.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:46.480 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:46.529 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:46.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:46.530 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:46.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi096.stderr:2024-09-21T02:09:46.572+0000 7f8a8d355580 -1 init, newargv = 0x7f8a7c0ad020 newargc=15 2024-09-21T02:09:46.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi096.stderr:ceph-fuse[37016]: starting ceph client 2024-09-21T02:09:46.581 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-21T02:09:46.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi096.stderr:ceph-fuse[37016]: starting fuse 2024-09-21T02:09:47.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:09:47.702 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:09:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:09:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:09:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:09:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:09:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:47.765 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:09:47.818 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-21T02:09:47.818 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-21T02:09:47.819 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-21T02:09:47.819 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-21T02:09:47.820 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-21T02:09:47.821 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-21T02:09:47.821 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:09:47.821 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:09:47.821 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.2")) 2024-09-21T02:09:47.821 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-21T02:09:47.948 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.2.37016.asok 2024-09-21T02:09:47.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:47.961 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.37016.asok 2024-09-21T02:09:47.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:47.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.37016.asok status 2024-09-21T02:09:48.146 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:09:48.146 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-21T02:09:48.146 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:09:48.146 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:09:48.146 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "2", 2024-09-21T02:09:48.147 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-21T02:09:48.147 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-21T02:09:48.147 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "37016", 2024-09-21T02:09:48.147 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-21T02:09:48.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:48.147 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-21T02:09:48.148 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-21T02:09:48.148 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4803, 2024-09-21T02:09:48.148 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-21T02:09:48.148 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-21T02:09:48.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4803 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1740499101 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:09:48.150 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1740499101 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4803 192.168.0.2:0/1740499101", 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/1740499101", 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 9, 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 26, 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-21T02:09:48.151 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:09:48.163 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:09:48.164 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:09:48.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 -- 172.21.15.96:0/968564274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664156bd0 msgr2=0x7fd6640cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/968564274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664156bd0 0x7fd6640cca70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd6640748b0 tx=0x7fd65401cd70 comp rx=0 tx=0).stop 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 -- 172.21.15.96:0/968564274 shutdown_connections 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/968564274 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd664164310 0x7fd664166730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/968564274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664156bd0 0x7fd6640cca70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 -- 172.21.15.96:0/968564274 >> 172.21.15.96:0/968564274 conn(0x7fd6640cf150 msgr2=0x7fd6640cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 -- 172.21.15.96:0/968564274 shutdown_connections 2024-09-21T02:09:48.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 -- 172.21.15.96:0/968564274 wait complete. 2024-09-21T02:09:48.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 Processor -- start 2024-09-21T02:09:48.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.323+0000 7fd66c0dc640 1 -- start start 2024-09-21T02:09:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6640b41a0 0x7fd6640c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 0x7fd66414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd664153d00 0x7fd66414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641698d0 con 0x7fd664164310 2024-09-21T02:09:48.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd664169750 con 0x7fd6640b41a0 2024-09-21T02:09:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd664169a50 con 0x7fd664153d00 2024-09-21T02:09:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 0x7fd66414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 0x7fd66414eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47744/0 (socket says 172.21.15.96:47744) 2024-09-21T02:09:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 -- 172.21.15.96:0/2122333641 learned_addr learned my addr 172.21.15.96:0/2122333641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669e78640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6640b41a0 0x7fd6640c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66a679640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd664153d00 0x7fd66414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 -- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd664153d00 msgr2=0x7fd66414f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd664153d00 0x7fd66414f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 -- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6640b41a0 msgr2=0x7fd6640c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6640b41a0 0x7fd6640c4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 -- 172.21.15.96:0/2122333641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6641f7760 con 0x7fd664164310 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd669677640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 0x7fd66414eba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd65800b3e0 tx=0x7fd65800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658015020 con 0x7fd664164310 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd658011920 con 0x7fd664164310 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6641f7a40 con 0x7fd664164310 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658011db0 con 0x7fd664164310 2024-09-21T02:09:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6641f8000 con 0x7fd664164310 2024-09-21T02:09:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.327+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6641f74b0 con 0x7fd664164310 2024-09-21T02:09:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.331+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd658018020 con 0x7fd664164310 2024-09-21T02:09:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.331+0000 7fd6627fc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd650073200 0x7fd650075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.331+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd658094770 con 0x7fd664164310 2024-09-21T02:09:48.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.331+0000 7fd669e78640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd650073200 0x7fd650075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:48.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.331+0000 7fd669e78640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd650073200 0x7fd650075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd6540af000 tx=0x7fd65401b5b0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:48.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.335+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd65801f020 con 0x7fd664164310 2024-09-21T02:09:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd6640cca70 con 0x7fd664164310 2024-09-21T02:09:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd6627fc640 1 -- 172.21.15.96:0/2122333641 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd65805ef70 con 0x7fd664164310 2024-09-21T02:09:48.552 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd650073200 msgr2=0x7fd650075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd650073200 0x7fd650075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd6540af000 tx=0x7fd65401b5b0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 msgr2=0x7fd66414eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 0x7fd66414eba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd65800b3e0 tx=0x7fd65800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 shutdown_connections 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd650073200 0x7fd650075710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd664153d00 0x7fd66414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd664164310 0x7fd66414eba0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 --2- 172.21.15.96:0/2122333641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6640b41a0 0x7fd6640c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 >> 172.21.15.96:0/2122333641 conn(0x7fd6640cf150 msgr2=0x7fd66415dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 shutdown_connections 2024-09-21T02:09:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:48.547+0000 7fd66c0dc640 1 -- 172.21.15.96:0/2122333641 wait complete. 2024-09-21T02:09:48.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:09:48.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:09:48.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-21T02:09:48.575 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:09:48.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:48.575 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:09:48.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:09:48.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:09:48.575 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:09:48.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:48.575 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-21T02:09:48.582 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:48.582 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:48.582 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42476sec preferred_lft 42476sec 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:11:74:1c:8f brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:09:48.584 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:48.584 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:09:48.584 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:48.584 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:48.584 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:09:48.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-09-21T02:09:48.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-09-21T02:09:48.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:09:48.585 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:48.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:48.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:48.715 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:09:48.724 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-21T02:09:48.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:48.726 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-21T02:09:48.783 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-21T02:09:48.783 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:09:48.784 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:48.784 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:48.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:09:48.785 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:09:48.785 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:09:48.785 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:48.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:48.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:48.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:48.902 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-21T02:09:48.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:48.951 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-09-21T02:09:48.998 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:48.998 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:48.998 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:48.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T02:09:48.999 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:49.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:49.119 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T02:09:49.119 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:49.119 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:49.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T02:09:49.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:09:49.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T02:09:49.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T02:09:49.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T02:09:49.120 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:49.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:49.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:49.227 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:49.227 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:49.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-21T02:09:49.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:09:49.227 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:49.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:49.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:49.330 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:09:49.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:49.330 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:49.381 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:09:49.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:49.382 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:49.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-09-21T02:09:49.482 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:49.531 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-09-21T02:09:49.531 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-09-21T02:09:49.531 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-09-21T02:09:49.531 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-09-21T02:09:49.532 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:49.533 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:49.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:49.534 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-09-21T02:09:49.583 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:09:49.583 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:09:49.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-09-21T02:09:49.642 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-09-21T02:09:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:09:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:09:49.693 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:09:49.693 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:09:49.693 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-09-21T02:09:49.693 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:49.693 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:49.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:49.694 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-09-21T02:09:49.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi122.stderr:2024-09-21T02:09:49.732+0000 7f2cc7d5c580 -1 init, newargv = 0x7f2cb80ad020 newargc=15 2024-09-21T02:09:49.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi122.stderr:ceph-fuse[34325]: starting ceph client 2024-09-21T02:09:49.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi122.stderr:ceph-fuse[34325]: starting fuse 2024-09-21T02:09:50.842 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-09-21T02:09:50.859 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:50.907 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-09-21T02:09:50.907 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-09-21T02:09:50.907 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-09-21T02:09:50.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:50.909 INFO:teuthology.orchestra.run.smithi122.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:50.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:50.910 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-09-21T02:09:50.959 INFO:teuthology.orchestra.run.smithi122.stdout:45 2024-09-21T02:09:50.959 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:09:50.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:09:50.959 DEBUG:teuthology.orchestra.run.smithi122:> import glob 2024-09-21T02:09:50.959 DEBUG:teuthology.orchestra.run.smithi122:> import re 2024-09-21T02:09:50.959 DEBUG:teuthology.orchestra.run.smithi122:> import os 2024-09-21T02:09:50.959 DEBUG:teuthology.orchestra.run.smithi122:> import subprocess 2024-09-21T02:09:50.959 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> def _find_admin_socket(client_name): 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> files = glob.glob(asok_path) 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> # Given a non-glob path, it better be there 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> if "*" not in asok_path: 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> assert(len(files) == 1) 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> return files[0] 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> for f in files: 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:09:50.960 DEBUG:teuthology.orchestra.run.smithi122:> contents = proc_f.read() 2024-09-21T02:09:50.961 DEBUG:teuthology.orchestra.run.smithi122:> if mountpoint in contents: 2024-09-21T02:09:50.961 DEBUG:teuthology.orchestra.run.smithi122:> return f 2024-09-21T02:09:50.961 DEBUG:teuthology.orchestra.run.smithi122:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:09:50.961 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:50.961 DEBUG:teuthology.orchestra.run.smithi122:> print(_find_admin_socket("client.1")) 2024-09-21T02:09:50.961 DEBUG:teuthology.orchestra.run.smithi122:> ' 2024-09-21T02:09:51.083 INFO:teuthology.orchestra.run.smithi122.stdout:/var/run/ceph/ceph-client.1.34325.asok 2024-09-21T02:09:51.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:51.095 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34325.asok 2024-09-21T02:09:51.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:51.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34325.asok status 2024-09-21T02:09:51.276 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T02:09:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": { 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "entity_id": "1", 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": "smithi122", 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "pid": "34325", 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "root": "/" 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_count": 0, 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_pinned_count": 0, 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4529, 2024-09-21T02:09:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "inst": { 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "name": { 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "client", 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "num": 4529 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3196555738 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-09-21T02:09:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3196555738 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "inst_str": "client.4529 192.168.0.1:0/3196555738", 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "addr_str": "192.168.0.1:0/3196555738", 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "inode_count": 1, 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_epoch": 9, 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch": 26, 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "blocklisted": false, 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs" 2024-09-21T02:09:51.279 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T02:09:51.292 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:09:51.292 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 -- 172.21.15.96:0/2105184979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980c53a0 msgr2=0x7f5f980cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 --2- 172.21.15.96:0/2105184979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980c53a0 0x7f5f980cf010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009bc0 tx=0x7f5f9001cb10 comp rx=0 tx=0).stop 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 -- 172.21.15.96:0/2105184979 shutdown_connections 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 --2- 172.21.15.96:0/2105184979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980c53a0 0x7f5f980cf010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 -- 172.21.15.96:0/2105184979 >> 172.21.15.96:0/2105184979 conn(0x7f5f9806eea0 msgr2=0x7f5f9806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 -- 172.21.15.96:0/2105184979 shutdown_connections 2024-09-21T02:09:51.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 -- 172.21.15.96:0/2105184979 wait complete. 2024-09-21T02:09:51.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.455+0000 7f5fa0128640 1 Processor -- start 2024-09-21T02:09:51.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- start start 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9806e2c0 0x7f5f981f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f980c53a0 0x7f5f9814b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 0x7f5f981530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9816ae80 con 0x7f5f980cec00 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9816ad00 con 0x7f5f9806e2c0 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9816b000 con 0x7f5f980c53a0 2024-09-21T02:09:51.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 0x7f5f981530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 0x7f5f981530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47750/0 (socket says 172.21.15.96:47750) 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 -- 172.21.15.96:0/55363671 learned_addr learned my addr 172.21.15.96:0/55363671 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9dec4640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9806e2c0 0x7f5f981f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9d6c3640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f980c53a0 0x7f5f9814b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 -- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f980c53a0 msgr2=0x7f5f9814b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f980c53a0 0x7f5f9814b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 -- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9806e2c0 msgr2=0x7f5f981f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9806e2c0 0x7f5f981f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 -- 172.21.15.96:0/55363671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f981fa930 con 0x7f5f980cec00 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9d6c3640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f980c53a0 0x7f5f9814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f9e6c5640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 0x7f5f981530e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b0040 tx=0x7f5f90005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900b2040 con 0x7f5f980cec00 2024-09-21T02:09:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f900bb070 con 0x7f5f980cec00 2024-09-21T02:09:51.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f981fa370 con 0x7f5f980cec00 2024-09-21T02:09:51.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90004650 con 0x7f5f980cec00 2024-09-21T02:09:51.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f980c71b0 con 0x7f5f980cec00 2024-09-21T02:09:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.459+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f980cf010 con 0x7f5f980cec00 2024-09-21T02:09:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.463+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5f900c0460 con 0x7f5f980cec00 2024-09-21T02:09:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.463+0000 7f5f8e7fc640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5f7c073200 0x7f5f7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.463+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5f9012f110 con 0x7f5f980cec00 2024-09-21T02:09:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.463+0000 7f5f9dec4640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5f7c073200 0x7f5f7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.463+0000 7f5f9dec4640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5f7c073200 0x7f5f7c075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f94000f70 tx=0x7f5f94009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.467+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5f981fa930 con 0x7f5f980cec00 2024-09-21T02:09:51.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.691+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f981fa100 con 0x7f5f980cec00 2024-09-21T02:09:51.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.691+0000 7f5f8e7fc640 1 -- 172.21.15.96:0/55363671 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5f90016790 con 0x7f5f980cec00 2024-09-21T02:09:51.697 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T02:09:51.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5f7c073200 msgr2=0x7f5f7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:51.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5f7c073200 0x7f5f7c075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f94000f70 tx=0x7f5f94009040 comp rx=0 tx=0).stop 2024-09-21T02:09:51.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 msgr2=0x7f5f981530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:51.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 0x7f5f981530e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b0040 tx=0x7f5f90005ae0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 shutdown_connections 2024-09-21T02:09:51.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5f7c073200 0x7f5f7c075710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f980cec00 0x7f5f981530e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f980c53a0 0x7f5f9814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 --2- 172.21.15.96:0/55363671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9806e2c0 0x7f5f981f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:51.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 >> 172.21.15.96:0/55363671 conn(0x7f5f9806eea0 msgr2=0x7f5f9806fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:51.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 shutdown_connections 2024-09-21T02:09:51.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:09:51.695+0000 7f5fa0128640 1 -- 172.21.15.96:0/55363671 wait complete. 2024-09-21T02:09:51.718 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:09:51.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:09:51.719 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T02:09:51.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:51.719 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-21T02:09:51.726 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:09:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42473sec preferred_lft 42473sec 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:11:74:1c:8f brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:09:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.729 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:09:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:09:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:51.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:51.729 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:09:51.783 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-21T02:09:51.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:51.785 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-21T02:09:51.835 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-21T02:09:51.835 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:09:51.835 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:09:51.835 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:09:51.835 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:09:51.835 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:09:51.836 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:09:51.836 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:51.836 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:51.836 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:09:51.836 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:09:51.836 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:51.836 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:51.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:09:51.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:09:51.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:09:51.836 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:51.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:51.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:51.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:51.960 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-21T02:09:51.965 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:09:51.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:51.965 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-09-21T02:09:52.013 INFO:teuthology.orchestra.run.smithi122.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T02:09:52.014 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:52.014 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:52.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:52.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T02:09:52.014 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:52.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:52.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:52.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:52.127 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T02:09:52.180 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether a2:75:1b:94:75:48 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::a075:1bff:fe94:7548/64 scope link 2024-09-21T02:09:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:52.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:09:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether a2:75:1b:94:75:48 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:09:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:09:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::a075:1bff:fe94:7548/64 scope link 2024-09-21T02:09:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:09:52.244 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T02:09:52.244 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:52.244 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:52.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T02:09:52.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T02:09:52.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T02:09:52.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T02:09:52.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T02:09:52.245 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:52.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:52.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:52.389 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:09:52.389 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-21T02:09:52.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.1 up 2024-09-21T02:09:52.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T02:09:52.389 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-21T02:09:52.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:09:52.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:52.450 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-21T02:09:52.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:52.450 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:52.500 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T02:09:52.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:09:52.502 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:52.549 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-09-21T02:09:52.608 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-09-21T02:09:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-09-21T02:09:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:52.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:52.662 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-09-21T02:09:52.711 INFO:teuthology.orchestra.run.smithi122.stdout:45 2024-09-21T02:09:52.711 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-21T02:09:52.711 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-21T02:09:52.756 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-09-21T02:09:52.770 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-09-21T02:09:52.819 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-09-21T02:09:52.820 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:52.821 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:52.822 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:52.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:52.823 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-09-21T02:09:52.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi122.stderr:2024-09-21T02:09:52.860+0000 7f2584006580 -1 init, newargv = 0x55892240a670 newargc=15 2024-09-21T02:09:52.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi122.stderr:ceph-fuse[34444]: starting ceph client 2024-09-21T02:09:52.867 INFO:teuthology.orchestra.run.smithi122.stdout:45 2024-09-21T02:09:52.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi122.stderr:ceph-fuse[34444]: starting fuse 2024-09-21T02:09:53.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-09-21T02:09:53.992 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:09:54.044 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-09-21T02:09:54.045 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-09-21T02:09:54.045 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-09-21T02:09:54.045 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-09-21T02:09:54.045 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-09-21T02:09:54.046 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:09:54.047 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:09:54.049 INFO:teuthology.orchestra.run.smithi122.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:54.049 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:54.049 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:09:54.049 INFO:teuthology.orchestra.run.smithi122.stdout:/home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:54.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.049 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-09-21T02:09:54.095 INFO:teuthology.orchestra.run.smithi122.stdout:45 2024-09-21T02:09:54.096 INFO:teuthology.orchestra.run.smithi122.stdout:46 2024-09-21T02:09:54.096 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-21T02:09:54.096 DEBUG:teuthology.orchestra.run.smithi122:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:09:54.096 DEBUG:teuthology.orchestra.run.smithi122:> import glob 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> import re 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> import os 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> import subprocess 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> def _find_admin_socket(client_name): 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> files = glob.glob(asok_path) 2024-09-21T02:09:54.097 DEBUG:teuthology.orchestra.run.smithi122:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> # Given a non-glob path, it better be there 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> if "*" not in asok_path: 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> assert(len(files) == 1) 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> return files[0] 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:54.098 DEBUG:teuthology.orchestra.run.smithi122:> for f in files: 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> contents = proc_f.read() 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> if mountpoint in contents: 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> return f 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:09:54.099 DEBUG:teuthology.orchestra.run.smithi122:> 2024-09-21T02:09:54.100 DEBUG:teuthology.orchestra.run.smithi122:> print(_find_admin_socket("client.3")) 2024-09-21T02:09:54.100 DEBUG:teuthology.orchestra.run.smithi122:> ' 2024-09-21T02:09:54.223 INFO:teuthology.orchestra.run.smithi122.stdout:/var/run/ceph/ceph-client.3.34444.asok 2024-09-21T02:09:54.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T02:09:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:09:54.235 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.34444.asok 2024-09-21T02:09:54.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.236 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.34444.asok status 2024-09-21T02:09:54.423 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": { 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "entity_id": "3", 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": "smithi122", 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "pid": "34444", 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: "root": "/" 2024-09-21T02:09:54.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_count": 0, 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_pinned_count": 0, 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4532, 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "inst": { 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "name": { 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "client", 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "num": 4532 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:54.425 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 2736961411 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-09-21T02:09:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:09:54.427 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 2736961411 2024-09-21T02:09:54.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T02:09:54.427 INFO:teuthology.orchestra.run.smithi122.stdout: "inst_str": "client.4532 192.168.0.2:0/2736961411", 2024-09-21T02:09:54.427 INFO:teuthology.orchestra.run.smithi122.stdout: "addr_str": "192.168.0.2:0/2736961411", 2024-09-21T02:09:54.427 INFO:teuthology.orchestra.run.smithi122.stdout: "inode_count": 1, 2024-09-21T02:09:54.427 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_epoch": 9, 2024-09-21T02:09:54.428 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch": 28, 2024-09-21T02:09:54.428 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:09:54.428 INFO:teuthology.orchestra.run.smithi122.stdout: "blocklisted": false, 2024-09-21T02:09:54.428 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs" 2024-09-21T02:09:54.428 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T02:09:54.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.439 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-21T02:09:54.439 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:54.447 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-21T02:09:54.447 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:54.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.448 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:09:54.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.507 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-21T02:09:54.507 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:54.557 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-21T02:09:54.558 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:54.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.558 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T02:09:54.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.616 DEBUG:teuthology.orchestra.run.smithi122:> stat --file-system '--printf=%T 2024-09-21T02:09:54.616 DEBUG:teuthology.orchestra.run.smithi122:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:54.623 INFO:teuthology.orchestra.run.smithi122.stdout:fuseblk 2024-09-21T02:09:54.624 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:54.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:09:54.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.683 DEBUG:teuthology.orchestra.run.smithi122:> stat --file-system '--printf=%T 2024-09-21T02:09:54.683 DEBUG:teuthology.orchestra.run.smithi122:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:54.732 INFO:teuthology.orchestra.run.smithi122.stdout:fuseblk 2024-09-21T02:09:54.732 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:54.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:09:54.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T02:09:54.787 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T02:10:05.091 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-09-21T02:10:05.091 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:10:05.092 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:10:05.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:10:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 -- 172.21.15.96:0/964239686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8073800 msgr2=0x7f4dc8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 --2- 172.21.15.96:0/964239686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8073800 0x7f4dc8073c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8155450 tx=0x7f4dc001d0d0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 -- 172.21.15.96:0/964239686 shutdown_connections 2024-09-21T02:10:05.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 --2- 172.21.15.96:0/964239686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8073800 0x7f4dc8073c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 --2- 172.21.15.96:0/964239686 >> v2:172.21.15.122:3300/0 conn(0x7f4dc806dea0 0x7f4dc806e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 -- 172.21.15.96:0/964239686 >> 172.21.15.96:0/964239686 conn(0x7f4dc806eea0 msgr2=0x7f4dc806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:05.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 -- 172.21.15.96:0/964239686 shutdown_connections 2024-09-21T02:10:05.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 -- 172.21.15.96:0/964239686 wait complete. 2024-09-21T02:10:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 Processor -- start 2024-09-21T02:10:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.247+0000 7f4dcedec640 1 -- start start 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4dc806dea0 0x7f4dc81fc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc8073800 0x7f4dc81f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 0x7f4dc8156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8167e90 con 0x7f4dc8155040 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8167d10 con 0x7f4dc8073800 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc8168010 con 0x7f4dc806dea0 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 0x7f4dc8156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 0x7f4dc8156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40936/0 (socket says 172.21.15.96:40936) 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 -- 172.21.15.96:0/3941150174 learned_addr learned my addr 172.21.15.96:0/3941150174 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dccb88640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4dc806dea0 0x7f4dc81fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc7fff640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc8073800 0x7f4dc81f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 -- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4dc806dea0 msgr2=0x7f4dc81fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4dc806dea0 0x7f4dc81fc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 -- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc8073800 msgr2=0x7f4dc81f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc8073800 0x7f4dc81f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 -- 172.21.15.96:0/3941150174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc8158cb0 con 0x7f4dc8155040 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dccb88640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4dc806dea0 0x7f4dc81fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc7fff640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc8073800 0x7f4dc81f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcd389640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 0x7f4dc8156740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001b780 tx=0x7f4dc00ac560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00b1040 con 0x7f4dc8155040 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dc00bc070 con 0x7f4dc8155040 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc8158f90 con 0x7f4dc8155040 2024-09-21T02:10:05.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00adc90 con 0x7f4dc8155040 2024-09-21T02:10:05.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc8159550 con 0x7f4dc8155040 2024-09-21T02:10:05.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc806e2b0 con 0x7f4dc8155040 2024-09-21T02:10:05.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4dc0003fe0 con 0x7f4dc8155040 2024-09-21T02:10:05.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.251+0000 7f4dc57fa640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4dac073200 0x7f4dac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.255+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f4dc012f070 con 0x7f4dc8155040 2024-09-21T02:10:05.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.255+0000 7f4dccb88640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4dac073200 0x7f4dac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.255+0000 7f4dccb88640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4dac073200 0x7f4dac075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806fe80 tx=0x7f4dbc008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:05.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.259+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4dc00a5e20 con 0x7f4dc8155040 2024-09-21T02:10:05.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.427+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dc814da90 con 0x7f4dc8155040 2024-09-21T02:10:05.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dc57fa640 1 -- 172.21.15.96:0/3941150174 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18985 (secure 0 0 0) 0x7f4dc806e7f0 con 0x7f4dc8155040 2024-09-21T02:10:05.435 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:05.435 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":28,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:51.588360+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:10:05.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4dac073200 msgr2=0x7f4dac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4dac073200 0x7f4dac075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806fe80 tx=0x7f4dbc008040 comp rx=0 tx=0).stop 2024-09-21T02:10:05.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 msgr2=0x7f4dc8156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 0x7f4dc8156740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001b780 tx=0x7f4dc00ac560 comp rx=0 tx=0).stop 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 shutdown_connections 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4dac073200 0x7f4dac075710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc8155040 0x7f4dc8156740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc8073800 0x7f4dc81f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 --2- 172.21.15.96:0/3941150174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4dc806dea0 0x7f4dc81fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 >> 172.21.15.96:0/3941150174 conn(0x7f4dc806eea0 msgr2=0x7f4dc81538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 shutdown_connections 2024-09-21T02:10:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.431+0000 7f4dcedec640 1 -- 172.21.15.96:0/3941150174 wait complete. 2024-09-21T02:10:05.455 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:09:00.544933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:09:08.825867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:09:09.856090+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T02:09:13.568880+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T02:10:05.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:10:05.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1e12e1640 1 -- 172.21.15.96:0/2893449318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1c8004d50 msgr2=0x7fb1c8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/2893449318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1c8004d50 0x7fb1c8005200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4001dc0 tx=0x7fb1d401cb30 comp rx=0 tx=0).stop 2024-09-21T02:10:05.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1db7fe640 1 -- 172.21.15.96:0/2893449318 reap_dead start 2024-09-21T02:10:05.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1e12e1640 1 -- 172.21.15.96:0/2893449318 shutdown_connections 2024-09-21T02:10:05.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1e12e1640 1 -- 172.21.15.96:0/2893449318 >> 172.21.15.96:0/2893449318 conn(0x7fb1dc0d1b10 msgr2=0x7fb1dc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:05.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1e12e1640 1 -- 172.21.15.96:0/2893449318 shutdown_connections 2024-09-21T02:10:05.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.611+0000 7fb1e12e1640 1 -- 172.21.15.96:0/2893449318 wait complete. 2024-09-21T02:10:05.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 Processor -- start 2024-09-21T02:10:05.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 -- start start 2024-09-21T02:10:05.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1c8004d50 0x7fb1dc1f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1dc1e6e30 0x7fb1dc15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 0x7fb1dc1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d3640 con 0x7fb1dc15e290 2024-09-21T02:10:05.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d34c0 con 0x7fb1c8004d50 2024-09-21T02:10:05.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d2460 con 0x7fb1dc1e6e30 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 0x7fb1dc1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 0x7fb1dc1f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40952/0 (socket says 172.21.15.96:40952) 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 -- 172.21.15.96:0/3969214049 learned_addr learned my addr 172.21.15.96:0/3969214049 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1d3fff640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1dc1e6e30 0x7fb1dc15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1daffd640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1c8004d50 0x7fb1dc1f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 -- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1dc1e6e30 msgr2=0x7fb1dc15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1dc1e6e30 0x7fb1dc15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 -- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1c8004d50 msgr2=0x7fb1dc1f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1c8004d50 0x7fb1dc1f8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 -- 172.21.15.96:0/3969214049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc1f9380 con 0x7fb1dc15e290 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1db7fe640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 0x7fb1dc1f5d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ab040 tx=0x7fb1d4006e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4003d90 con 0x7fb1dc15e290 2024-09-21T02:10:05.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1d40060d0 con 0x7fb1dc15e290 2024-09-21T02:10:05.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40064e0 con 0x7fb1dc15e290 2024-09-21T02:10:05.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc1f9660 con 0x7fb1dc15e290 2024-09-21T02:10:05.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.615+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc1f5850 con 0x7fb1dc15e290 2024-09-21T02:10:05.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.619+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc15fdb0 con 0x7fb1dc15e290 2024-09-21T02:10:05.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.619+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb1d4006680 con 0x7fb1dc15e290 2024-09-21T02:10:05.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.619+0000 7fb1d37fe640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb1bc073230 0x7fb1bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.619+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb1d412e2c0 con 0x7fb1dc15e290 2024-09-21T02:10:05.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.619+0000 7fb1daffd640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb1bc073230 0x7fb1bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.619+0000 7fb1daffd640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb1bc073230 0x7fb1bc075740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0018b0 tx=0x7fb1cc001e40 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:05.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.623+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb1d40bf030 con 0x7fb1dc15e290 2024-09-21T02:10:05.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.807+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1dc15f680 con 0x7fb1dc15e290 2024-09-21T02:10:05.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.807+0000 7fb1d37fe640 1 -- 172.21.15.96:0/3969214049 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fb1d4016890 con 0x7fb1dc15e290 2024-09-21T02:10:05.812 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb1bc073230 msgr2=0x7fb1bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb1bc073230 0x7fb1bc075740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0018b0 tx=0x7fb1cc001e40 comp rx=0 tx=0).stop 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 msgr2=0x7fb1dc1f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 0x7fb1dc1f5d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ab040 tx=0x7fb1d4006e80 comp rx=0 tx=0).stop 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 shutdown_connections 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb1bc073230 0x7fb1bc075740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc15e290 0x7fb1dc1f5d70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1dc1e6e30 0x7fb1dc15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 --2- 172.21.15.96:0/3969214049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1c8004d50 0x7fb1dc1f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 >> 172.21.15.96:0/3969214049 conn(0x7fb1dc0d1b10 msgr2=0x7fb1dc0b9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 shutdown_connections 2024-09-21T02:10:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.811+0000 7fb1e12e1640 1 -- 172.21.15.96:0/3969214049 wait complete. 2024-09-21T02:10:05.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:10:05.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 -- 172.21.15.96:0/3006082991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb844071b90 msgr2=0x7fb8440cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/3006082991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb844071b90 0x7fb8440cf760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009c80 tx=0x7fb83c01cd10 comp rx=0 tx=0).stop 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 -- 172.21.15.96:0/3006082991 shutdown_connections 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/3006082991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb844071b90 0x7fb8440cf760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/3006082991 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb844074500 0x7fb84415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 -- 172.21.15.96:0/3006082991 >> 172.21.15.96:0/3006082991 conn(0x7fb8440d13d0 msgr2=0x7fb8440734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 -- 172.21.15.96:0/3006082991 shutdown_connections 2024-09-21T02:10:05.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.983+0000 7fb84a8d0640 1 -- 172.21.15.96:0/3006082991 wait complete. 2024-09-21T02:10:05.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 Processor -- start 2024-09-21T02:10:05.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- start start 2024-09-21T02:10:05.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb844074500 0x7fb84414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 0x7fb8441f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8441fa270 0x7fb84415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb844169a90 con 0x7fb8440cdbb0 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb844169910 con 0x7fb844074500 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb844169c10 con 0x7fb8441fa270 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 0x7fb8441f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 0x7fb8441f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40968/0 (socket says 172.21.15.96:40968) 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 -- 172.21.15.96:0/1487665443 learned_addr learned my addr 172.21.15.96:0/1487665443 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb843fff640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb844074500 0x7fb84414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb848e6d640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8441fa270 0x7fb84415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 -- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8441fa270 msgr2=0x7fb84415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8441fa270 0x7fb84415b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 -- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb844074500 msgr2=0x7fb84414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb844074500 0x7fb84414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 -- 172.21.15.96:0/1487665443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8441fa680 con 0x7fb8440cdbb0 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb848e6d640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8441fa270 0x7fb84415b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb8437fe640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 0x7fb8441f9d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c3f0 tx=0x7fb82c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c017070 con 0x7fb8440cdbb0 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb82c0053c0 con 0x7fb8440cdbb0 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84413ae60 con 0x7fb8440cdbb0 2024-09-21T02:10:05.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c003860 con 0x7fb8440cdbb0 2024-09-21T02:10:05.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84413b420 con 0x7fb8440cdbb0 2024-09-21T02:10:05.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.987+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8440cf760 con 0x7fb8440cdbb0 2024-09-21T02:10:05.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.991+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb82c01a070 con 0x7fb8440cdbb0 2024-09-21T02:10:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.991+0000 7fb840ff9640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb824073200 0x7fb824075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.991+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb82c093920 con 0x7fb8440cdbb0 2024-09-21T02:10:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.991+0000 7fb843fff640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb824073200 0x7fb824075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.991+0000 7fb843fff640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb824073200 0x7fb824075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb8340058d0 tx=0x7fb834005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:05.995+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb8441fa680 con 0x7fb8440cdbb0 2024-09-21T02:10:06.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.167+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8440cd080 con 0x7fb8440cdbb0 2024-09-21T02:10:06.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.167+0000 7fb840ff9640 1 -- 172.21.15.96:0/1487665443 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fb82c05e540 con 0x7fb8440cdbb0 2024-09-21T02:10:06.173 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb824073200 msgr2=0x7fb824075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb824073200 0x7fb824075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb8340058d0 tx=0x7fb834005860 comp rx=0 tx=0).stop 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 msgr2=0x7fb8441f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 0x7fb8441f9d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c3f0 tx=0x7fb82c00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 shutdown_connections 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb824073200 0x7fb824075710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8441fa270 0x7fb84415b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8440cdbb0 0x7fb8441f9d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 --2- 172.21.15.96:0/1487665443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb844074500 0x7fb84414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 >> 172.21.15.96:0/1487665443 conn(0x7fb8440d13d0 msgr2=0x7fb8440b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 shutdown_connections 2024-09-21T02:10:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.171+0000 7fb84a8d0640 1 -- 172.21.15.96:0/1487665443 wait complete. 2024-09-21T02:10:06.193 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:10:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/2077793050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff494004e70 msgr2=0x7ff4940052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/2077793050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff494004e70 0x7ff4940052c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4001dc0 tx=0x7ff4a401cb30 comp rx=0 tx=0).stop 2024-09-21T02:10:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4aad9d640 1 -- 172.21.15.96:0/2077793050 reap_dead start 2024-09-21T02:10:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/2077793050 shutdown_connections 2024-09-21T02:10:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/2077793050 >> 172.21.15.96:0/2077793050 conn(0x7ff4ac06eea0 msgr2=0x7ff4ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/2077793050 shutdown_connections 2024-09-21T02:10:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/2077793050 wait complete. 2024-09-21T02:10:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 Processor -- start 2024-09-21T02:10:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.343+0000 7ff4b0f4e640 1 -- start start 2024-09-21T02:10:06.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff494004e70 0x7ff4ac1f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 0x7ff4ac1e9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4ac1e9870 0x7ff4ac0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15cbb0 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15ca30 con 0x7ff4ac1e9870 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac0c4580 con 0x7ff494004e70 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 0x7ff4ac1e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 0x7ff4ac1e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40986/0 (socket says 172.21.15.96:40986) 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 -- 172.21.15.96:0/904898662 learned_addr learned my addr 172.21.15.96:0/904898662 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4aa59c640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff494004e70 0x7ff4ac1f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4aad9d640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4ac1e9870 0x7ff4ac0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 -- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff494004e70 msgr2=0x7ff4ac1f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff494004e70 0x7ff4ac1f9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 -- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4ac1e9870 msgr2=0x7ff4ac0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4ac1e9870 0x7ff4ac0ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 -- 172.21.15.96:0/904898662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac1e9c80 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4a9d9b640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 0x7ff4ac1e9330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac1601e0 tx=0x7ff4a000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0012850 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4a0012ae0 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac1fa030 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a001a400 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1f5f00 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac0cf010 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff4a000d030 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff49affd640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff48c073230 0x7ff48c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.347+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff4a0097850 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.351+0000 7ff4aa59c640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff48c073230 0x7ff48c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.351+0000 7ff4aa59c640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff48c073230 0x7ff48c075740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff49c005fd0 tx=0x7ff49c005f40 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:06.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.355+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff4ac1e9c80 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.511+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4ac15f680 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.511+0000 7ff49affd640 1 -- 172.21.15.96:0/904898662 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7ff4a0016030 con 0x7ff4ac0cbfc0 2024-09-21T02:10:06.519 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff48c073230 msgr2=0x7ff48c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff48c073230 0x7ff48c075740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff49c005fd0 tx=0x7ff49c005f40 comp rx=0 tx=0).stop 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 msgr2=0x7ff4ac1e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 0x7ff4ac1e9330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac1601e0 tx=0x7ff4a000e6b0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 shutdown_connections 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff48c073230 0x7ff48c075740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4ac1e9870 0x7ff4ac0ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4ac0cbfc0 0x7ff4ac1e9330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 --2- 172.21.15.96:0/904898662 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff494004e70 0x7ff4ac1f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 >> 172.21.15.96:0/904898662 conn(0x7ff4ac06eea0 msgr2=0x7ff4ac0c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 shutdown_connections 2024-09-21T02:10:06.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.515+0000 7ff4b0f4e640 1 -- 172.21.15.96:0/904898662 wait complete. 2024-09-21T02:10:06.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T02:10:06.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 -- 172.21.15.96:0/832553324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 msgr2=0x7fbb940cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/832553324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940cf010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406a1c0 tx=0x7fbb8c01caf0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 -- 172.21.15.96:0/832553324 shutdown_connections 2024-09-21T02:10:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/832553324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940cf010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/832553324 >> v2:172.21.15.122:3300/0 conn(0x7fbb9406e2c0 0x7fbb940c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 -- 172.21.15.96:0/832553324 >> 172.21.15.96:0/832553324 conn(0x7fbb9406eea0 msgr2=0x7fbb9406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 -- 172.21.15.96:0/832553324 shutdown_connections 2024-09-21T02:10:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 -- 172.21.15.96:0/832553324 wait complete. 2024-09-21T02:10:06.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 Processor -- start 2024-09-21T02:10:06.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.679+0000 7fbb9abe0640 1 -- start start 2024-09-21T02:10:06.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb9406e2c0 0x7fbb940c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb940c4ac0 0x7fbb9413aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94169c60 con 0x7fbb940cec00 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94169ae0 con 0x7fbb9406e2c0 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbb94169de0 con 0x7fbb940c4ac0 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41010/0 (socket says 172.21.15.96:41010) 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 -- 172.21.15.96:0/1636301919 learned_addr learned my addr 172.21.15.96:0/1636301919 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9897c640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb9406e2c0 0x7fbb940c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb93fff640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb940c4ac0 0x7fbb9413aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 -- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb940c4ac0 msgr2=0x7fbb9413aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb940c4ac0 0x7fbb9413aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 -- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb9406e2c0 msgr2=0x7fbb940c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb9406e2c0 0x7fbb940c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 -- 172.21.15.96:0/1636301919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb940c8c60 con 0x7fbb940cec00 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb93fff640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb940c4ac0 0x7fbb9413aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9917d640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940c9a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009b10 tx=0x7fbb8c0a5850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0bb070 con 0x7fbb940cec00 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb8c004e00 con 0x7fbb940cec00 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b2040 con 0x7fbb940cec00 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb940c8f40 con 0x7fbb940cec00 2024-09-21T02:10:06.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb940c9500 con 0x7fbb940cec00 2024-09-21T02:10:06.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb94073310 con 0x7fbb940cec00 2024-09-21T02:10:06.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb8c0be070 con 0x7fbb940cec00 2024-09-21T02:10:06.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb917fa640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbb80073200 0x7fbb80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:06.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.683+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fbb8c12e440 con 0x7fbb940cec00 2024-09-21T02:10:06.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.687+0000 7fbb9897c640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbb80073200 0x7fbb80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:06.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.687+0000 7fbb9897c640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbb80073200 0x7fbb80075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb840058d0 tx=0x7fbb84005d90 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:06.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.691+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbb8c016770 con 0x7fbb940cec00 2024-09-21T02:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.859+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbb940c4560 con 0x7fbb940cec00 2024-09-21T02:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.859+0000 7fbb917fa640 1 -- 172.21.15.96:0/1636301919 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fbb8c0f9060 con 0x7fbb940cec00 2024-09-21T02:10:06.865 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbb80073200 msgr2=0x7fbb80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbb80073200 0x7fbb80075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb840058d0 tx=0x7fbb84005d90 comp rx=0 tx=0).stop 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 msgr2=0x7fbb940c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940c9a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009b10 tx=0x7fbb8c0a5850 comp rx=0 tx=0).stop 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 shutdown_connections 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fbb80073200 0x7fbb80075710 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb940cec00 0x7fbb940c9a20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb940c4ac0 0x7fbb9413aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 --2- 172.21.15.96:0/1636301919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb9406e2c0 0x7fbb940c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 >> 172.21.15.96:0/1636301919 conn(0x7fbb9406eea0 msgr2=0x7fbb940cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 shutdown_connections 2024-09-21T02:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:06.863+0000 7fbb9abe0640 1 -- 172.21.15.96:0/1636301919 wait complete. 2024-09-21T02:10:06.886 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 -- 172.21.15.96:0/4077734611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 msgr2=0x7fb03015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 --2- 172.21.15.96:0/4077734611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb03015bea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0300cca40 tx=0x7fb02401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 -- 172.21.15.96:0/4077734611 shutdown_connections 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 --2- 172.21.15.96:0/4077734611 >> v2:172.21.15.122:3300/0 conn(0x7fb03015c3e0 0x7fb0300cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 --2- 172.21.15.96:0/4077734611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb03015bea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 --2- 172.21.15.96:0/4077734611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0300ced00 0x7fb03006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 -- 172.21.15.96:0/4077734611 >> 172.21.15.96:0/4077734611 conn(0x7fb03006eea0 msgr2=0x7fb03006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 -- 172.21.15.96:0/4077734611 shutdown_connections 2024-09-21T02:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 -- 172.21.15.96:0/4077734611 wait complete. 2024-09-21T02:10:07.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.027+0000 7fb03638f640 1 Processor -- start 2024-09-21T02:10:07.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- start start 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb030154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0300ced00 0x7fb0301581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb03015c3e0 0x7fb0301586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300caae0 con 0x7fb03006e3e0 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300ca960 con 0x7fb03015c3e0 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb0300cac60 con 0x7fb0300ced00 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb030154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb030154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56718/0 (socket says 172.21.15.96:56718) 2024-09-21T02:10:07.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 -- 172.21.15.96:0/3228161033 learned_addr learned my addr 172.21.15.96:0/3228161033 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03492c640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb03015c3e0 0x7fb0301586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02f7fe640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0300ced00 0x7fb0301581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 -- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0300ced00 msgr2=0x7fb0301581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0300ced00 0x7fb0301581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 -- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb03015c3e0 msgr2=0x7fb0301586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb03015c3e0 0x7fb0301586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 -- 172.21.15.96:0/3228161033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0301559a0 con 0x7fb03006e3e0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03492c640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb03015c3e0 0x7fb0301586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02ffff640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb030154ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c9a0 tx=0x7fb02000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020007bb0 con 0x7fb03006e3e0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb020007e40 con 0x7fb03006e3e0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03015b220 con 0x7fb03006e3e0 2024-09-21T02:10:07.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020005260 con 0x7fb03006e3e0 2024-09-21T02:10:07.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb030164610 con 0x7fb03006e3e0 2024-09-21T02:10:07.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.031+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0300cb180 con 0x7fb03006e3e0 2024-09-21T02:10:07.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.035+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb020005400 con 0x7fb03006e3e0 2024-09-21T02:10:07.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.035+0000 7fb02cff9640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb01c073230 0x7fb01c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.035+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb020094a00 con 0x7fb03006e3e0 2024-09-21T02:10:07.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.035+0000 7fb02f7fe640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb01c073230 0x7fb01c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.035+0000 7fb02f7fe640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb01c073230 0x7fb01c075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0240af000 tx=0x7fb02401b100 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:07.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.039+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb020014030 con 0x7fb03006e3e0 2024-09-21T02:10:07.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.271+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7fb03006af50 con 0x7fb03006e3e0 2024-09-21T02:10:07.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.599+0000 7fb02cff9640 1 -- 172.21.15.96:0/3228161033 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v69) ==== 123+0+0 (secure 0 0 0) 0x7fb02005f620 con 0x7fb03006e3e0 2024-09-21T02:10:07.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb01c073230 msgr2=0x7fb01c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb01c073230 0x7fb01c075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0240af000 tx=0x7fb02401b100 comp rx=0 tx=0).stop 2024-09-21T02:10:07.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 msgr2=0x7fb030154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb030154ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c9a0 tx=0x7fb02000ce60 comp rx=0 tx=0).stop 2024-09-21T02:10:07.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 shutdown_connections 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb01c073230 0x7fb01c075740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb03015c3e0 0x7fb0301586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0300ced00 0x7fb0301581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 --2- 172.21.15.96:0/3228161033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb03006e3e0 0x7fb030154ad0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 >> 172.21.15.96:0/3228161033 conn(0x7fb03006eea0 msgr2=0x7fb03006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 shutdown_connections 2024-09-21T02:10:07.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.603+0000 7fb03638f640 1 -- 172.21.15.96:0/3228161033 wait complete. 2024-09-21T02:10:07.631 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:10:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 -- 172.21.15.96:0/202981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 msgr2=0x7f148c0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 --2- 172.21.15.96:0/202981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c0cc780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f147c009c80 tx=0x7f147c01cd10 comp rx=0 tx=0).stop 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 -- 172.21.15.96:0/202981118 shutdown_connections 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 --2- 172.21.15.96:0/202981118 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f148c06ffb0 0x7f148c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 --2- 172.21.15.96:0/202981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c0cc780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 -- 172.21.15.96:0/202981118 >> 172.21.15.96:0/202981118 conn(0x7f148c0cdbb0 msgr2=0x7f148c0cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 -- 172.21.15.96:0/202981118 shutdown_connections 2024-09-21T02:10:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.771+0000 7f1490946640 1 -- 172.21.15.96:0/202981118 wait complete. 2024-09-21T02:10:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 Processor -- start 2024-09-21T02:10:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- start start 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f148c06dfa0 0x7f148c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148c06ffb0 0x7f148c06ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f148c168d50 con 0x7f148c070f80 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f148c168bd0 con 0x7f148c06ffb0 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f148c168ed0 con 0x7f148c06dfa0 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c06d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56736/0 (socket says 172.21.15.96:56736) 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 -- 172.21.15.96:0/3045473394 learned_addr learned my addr 172.21.15.96:0/3045473394 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1489d9b640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148c06ffb0 0x7f148c06ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148a59c640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f148c06dfa0 0x7f148c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 -- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f148c06dfa0 msgr2=0x7f148c15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f148c06dfa0 0x7f148c15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 -- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148c06ffb0 msgr2=0x7f148c06ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148c06ffb0 0x7f148c06ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 -- 172.21.15.96:0/3045473394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c1f6b70 con 0x7f148c070f80 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1489d9b640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148c06ffb0 0x7f148c06ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f148ad9d640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c06d3d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f148400c3c0 tx=0x7f148400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484017070 con 0x7f148c070f80 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1484003600 con 0x7f148c070f80 2024-09-21T02:10:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c1faa60 con 0x7f148c070f80 2024-09-21T02:10:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484003a50 con 0x7f148c070f80 2024-09-21T02:10:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c1f70c0 con 0x7f148c070f80 2024-09-21T02:10:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.775+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c0703c0 con 0x7f148c070f80 2024-09-21T02:10:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.779+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f148401a070 con 0x7f148c070f80 2024-09-21T02:10:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.779+0000 7f1482ffd640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f146c073230 0x7f146c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.779+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1484093710 con 0x7f148c070f80 2024-09-21T02:10:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.779+0000 7f148a59c640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f146c073230 0x7f146c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.779+0000 7f148a59c640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f146c073230 0x7f146c075740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f147c0af000 tx=0x7f147c01b550 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.783+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f148c070900 con 0x7f148c070f80 2024-09-21T02:10:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.955+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f148c1fac30 con 0x7f148c070f80 2024-09-21T02:10:07.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.955+0000 7f1482ffd640 1 -- 172.21.15.96:0/3045473394 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f148405e3c0 con 0x7f148c070f80 2024-09-21T02:10:07.962 INFO:teuthology.orchestra.run.smithi096.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:10:07.962 INFO:teuthology.orchestra.run.smithi096.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:10:07.962 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:10:07.962 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced debug_client 20/20 2024-09-21T02:10:07.962 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3524.994876 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5166.000889 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3401.629435 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3663.438923 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4015.966326 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4135.580627 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5936.331421 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5058.300607 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced debug_mds 20/20 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:10:07.963 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced debug_ms 2/2 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds dev mds_debug_frag true 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds dev mds_verify_scatter true 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced client_mount_timeout 600 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced debug_client 20/20 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced debug_ms 1/1 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:10:07.964 INFO:teuthology.orchestra.run.smithi096.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:10:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f146c073230 msgr2=0x7f146c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f146c073230 0x7f146c075740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f147c0af000 tx=0x7f147c01b550 comp rx=0 tx=0).stop 2024-09-21T02:10:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 msgr2=0x7f148c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c06d3d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f148400c3c0 tx=0x7f148400c880 comp rx=0 tx=0).stop 2024-09-21T02:10:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 shutdown_connections 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f146c073230 0x7f146c075740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f148c070f80 0x7f148c06d3d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148c06ffb0 0x7f148c06ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 --2- 172.21.15.96:0/3045473394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f148c06dfa0 0x7f148c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 >> 172.21.15.96:0/3045473394 conn(0x7f148c0cdbb0 msgr2=0x7f148c0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 shutdown_connections 2024-09-21T02:10:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:07.959+0000 7f1490946640 1 -- 172.21.15.96:0/3045473394 wait complete. 2024-09-21T02:10:07.983 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:10:08.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 -- 172.21.15.96:0/1657855163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b8006e2c0 msgr2=0x7f1b800c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 --2- 172.21.15.96:0/1657855163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b8006e2c0 0x7f1b800c4580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009bc0 tx=0x7f1b7001cb10 comp rx=0 tx=0).stop 2024-09-21T02:10:08.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 -- 172.21.15.96:0/1657855163 shutdown_connections 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 --2- 172.21.15.96:0/1657855163 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b800c4ac0 0x7f1b800c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 --2- 172.21.15.96:0/1657855163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b8006e2c0 0x7f1b800c4580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 -- 172.21.15.96:0/1657855163 >> 172.21.15.96:0/1657855163 conn(0x7f1b8006eea0 msgr2=0x7f1b8006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 -- 172.21.15.96:0/1657855163 shutdown_connections 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.131+0000 7f1b8849a640 1 -- 172.21.15.96:0/1657855163 wait complete. 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 Processor -- start 2024-09-21T02:10:08.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- start start 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8006e2c0 0x7f1b80167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b800c4ac0 0x7f1b800c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 0x7f1b800b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800cf2b0 con 0x7f1b800cdbf0 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800cf130 con 0x7f1b8006e2c0 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800cf430 con 0x7f1b800c4ac0 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 0x7f1b800b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 0x7f1b800b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56758/0 (socket says 172.21.15.96:56758) 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 -- 172.21.15.96:0/2319505014 learned_addr learned my addr 172.21.15.96:0/2319505014 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86236640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8006e2c0 0x7f1b80167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b85a35640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b800c4ac0 0x7f1b800c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 -- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b800c4ac0 msgr2=0x7f1b800c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b800c4ac0 0x7f1b800c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 -- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8006e2c0 msgr2=0x7f1b80167bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8006e2c0 0x7f1b80167bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 -- 172.21.15.96:0/2319505014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b801f99d0 con 0x7f1b800cdbf0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86236640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8006e2c0 0x7f1b80167bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b86a37640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 0x7f1b800b4c30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c3c0 tx=0x7f1b7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78011790 con 0x7f1b800cdbf0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b78017070 con 0x7f1b800cdbf0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b801f8c10 con 0x7f1b800cdbf0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78011ce0 con 0x7f1b800cdbf0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b801f91d0 con 0x7f1b800cdbf0 2024-09-21T02:10:08.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.135+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b800c4580 con 0x7f1b800cdbf0 2024-09-21T02:10:08.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.139+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1b7801c460 con 0x7f1b800cdbf0 2024-09-21T02:10:08.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.139+0000 7f1b7effd640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1b6c073200 0x7f1b6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.139+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1b780943e0 con 0x7f1b800cdbf0 2024-09-21T02:10:08.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.139+0000 7f1b86236640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1b6c073200 0x7f1b6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.139+0000 7f1b86236640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1b6c073200 0x7f1b6c075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1b700af000 tx=0x7f1b7001b350 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:08.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.143+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1b7805f090 con 0x7f1b800cdbf0 2024-09-21T02:10:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.315+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1b80159a30 con 0x7f1b800cdbf0 2024-09-21T02:10:08.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.319+0000 7f1b7effd640 1 -- 172.21.15.96:0/2319505014 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f1b7805e990 con 0x7f1b800cdbf0 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1b6c073200 msgr2=0x7f1b6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1b6c073200 0x7f1b6c075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1b700af000 tx=0x7f1b7001b350 comp rx=0 tx=0).stop 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 msgr2=0x7f1b800b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 0x7f1b800b4c30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c3c0 tx=0x7f1b7800c880 comp rx=0 tx=0).stop 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 shutdown_connections 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1b6c073200 0x7f1b6c075710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b800cdbf0 0x7f1b800b4c30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b800c4ac0 0x7f1b800c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 --2- 172.21.15.96:0/2319505014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8006e2c0 0x7f1b80167bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 >> 172.21.15.96:0/2319505014 conn(0x7f1b8006eea0 msgr2=0x7f1b80153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:08.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 shutdown_connections 2024-09-21T02:10:08.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.323+0000 7f1b8849a640 1 -- 172.21.15.96:0/2319505014 wait complete. 2024-09-21T02:10:08.347 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:10:08.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:10:08.408 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:10:08.487 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:10:08.488 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:10:08.488 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:08.494 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:10:08.494 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:08.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 -- 172.21.15.96:0/4000371860 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c006e5b0 msgr2=0x7f33c00cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 --2- 172.21.15.96:0/4000371860 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c006e5b0 0x7f33c00cab60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33b800acd0 tx=0x7f33b801deb0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 -- 172.21.15.96:0/4000371860 shutdown_connections 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 --2- 172.21.15.96:0/4000371860 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c006e5b0 0x7f33c00cab60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 --2- 172.21.15.96:0/4000371860 >> v2:172.21.15.122:3300/0 conn(0x7f33c0155240 0x7f33c0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 -- 172.21.15.96:0/4000371860 >> 172.21.15.96:0/4000371860 conn(0x7f33c006eea0 msgr2=0x7f33c006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 -- 172.21.15.96:0/4000371860 shutdown_connections 2024-09-21T02:10:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 -- 172.21.15.96:0/4000371860 wait complete. 2024-09-21T02:10:08.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 Processor -- start 2024-09-21T02:10:08.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.743+0000 7f33c8684640 1 -- start start 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c0073970 0x7f33c014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33c0155240 0x7f33c0151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 0x7f33c0152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00cbe70 con 0x7f33c014b280 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00cbcf0 con 0x7f33c0155240 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f33c0167b70 con 0x7f33c0073970 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 0x7f33c0152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 0x7f33c0152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56762/0 (socket says 172.21.15.96:56762) 2024-09-21T02:10:08.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 -- 172.21.15.96:0/4002020066 learned_addr learned my addr 172.21.15.96:0/4002020066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c5c1f640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33c0155240 0x7f33c0151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6420640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c0073970 0x7f33c014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 -- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c0073970 msgr2=0x7f33c014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c0073970 0x7f33c014ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 -- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33c0155240 msgr2=0x7f33c0151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33c0155240 0x7f33c0151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 -- 172.21.15.96:0/4002020066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c014c000 con 0x7f33c014b280 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c5c1f640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33c0155240 0x7f33c0151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6420640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c0073970 0x7f33c014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6c21640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 0x7f33c0152260 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33b800ac20 tx=0x7f33b8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80bd070 con 0x7f33c014b280 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b80b3040 con 0x7f33c014b280 2024-09-21T02:10:08.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c014c2e0 con 0x7f33c014b280 2024-09-21T02:10:08.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80b2070 con 0x7f33c014b280 2024-09-21T02:10:08.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c014c8a0 con 0x7f33c014b280 2024-09-21T02:10:08.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c00cab60 con 0x7f33c014b280 2024-09-21T02:10:08.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33b80ad6f0 con 0x7f33c014b280 2024-09-21T02:10:08.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f33a4073200 0x7f33a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:08.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f33b812eef0 con 0x7f33c014b280 2024-09-21T02:10:08.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.747+0000 7f33c6420640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f33a4073200 0x7f33a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:08.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.751+0000 7f33c6420640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f33a4073200 0x7f33a4075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009ac0 tx=0x7f33bc009260 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:08.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.755+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33b8017aa0 con 0x7f33c014b280 2024-09-21T02:10:08.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.935+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33c006e9d0 con 0x7f33c014b280 2024-09-21T02:10:08.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33b6ffd640 1 -- 172.21.15.96:0/4002020066 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f33b80f9a90 con 0x7f33c014b280 2024-09-21T02:10:08.943 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f33a4073200 msgr2=0x7f33a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f33a4073200 0x7f33a4075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009ac0 tx=0x7f33bc009260 comp rx=0 tx=0).stop 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 msgr2=0x7f33c0152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 0x7f33c0152260 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33b800ac20 tx=0x7f33b8003f60 comp rx=0 tx=0).stop 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 shutdown_connections 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f33a4073200 0x7f33a4075710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33c014b280 0x7f33c0152260 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33c0155240 0x7f33c0151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 --2- 172.21.15.96:0/4002020066 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33c0073970 0x7f33c014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 >> 172.21.15.96:0/4002020066 conn(0x7f33c006eea0 msgr2=0x7f33c00b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 shutdown_connections 2024-09-21T02:10:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:08.939+0000 7f33c8684640 1 -- 172.21.15.96:0/4002020066 wait complete. 2024-09-21T02:10:08.966 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:10:08.966 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:08.972 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:09.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 -- 172.21.15.96:0/220644734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 msgr2=0x7f5ef4162d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 --2- 172.21.15.96:0/220644734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef4162d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40d5350 tx=0x7f5eec01cd60 comp rx=0 tx=0).stop 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 -- 172.21.15.96:0/220644734 shutdown_connections 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 --2- 172.21.15.96:0/220644734 >> v2:172.21.15.122:3300/0 conn(0x7f5ef415f7d0 0x7f5ef4162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 --2- 172.21.15.96:0/220644734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef4162d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 --2- 172.21.15.96:0/220644734 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ef40cf710 0x7f5ef415f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 -- 172.21.15.96:0/220644734 >> 172.21.15.96:0/220644734 conn(0x7f5ef40d1b10 msgr2=0x7f5ef40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 -- 172.21.15.96:0/220644734 shutdown_connections 2024-09-21T02:10:09.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.151+0000 7f5efc02b640 1 -- 172.21.15.96:0/220644734 wait complete. 2024-09-21T02:10:09.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 Processor -- start 2024-09-21T02:10:09.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 -- start start 2024-09-21T02:10:09.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ef40cf710 0x7f5ef4160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ef415f7d0 0x7f5ef414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef40ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef40d2df0 con 0x7f5ef4162950 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef40d2c70 con 0x7f5ef40cf710 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef40d2f70 con 0x7f5ef415f7d0 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef40ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef40ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56770/0 (socket says 172.21.15.96:56770) 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 -- 172.21.15.96:0/3225173008 learned_addr learned my addr 172.21.15.96:0/3225173008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5ef9dc7640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ef40cf710 0x7f5ef4160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5ef95c6640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ef415f7d0 0x7f5ef414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 -- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ef415f7d0 msgr2=0x7f5ef414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ef415f7d0 0x7f5ef414fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 -- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ef40cf710 msgr2=0x7f5ef4160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ef40cf710 0x7f5ef4160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 -- 172.21.15.96:0/3225173008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef41626b0 con 0x7f5ef4162950 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5ef9dc7640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ef40cf710 0x7f5ef4160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efa5c8640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef40ce150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5eec01b410 tx=0x7f5eec0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0bb070 con 0x7f5ef4162950 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5eec004d40 con 0x7f5ef4162950 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef4151530 con 0x7f5ef4162950 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0b2040 con 0x7f5ef4162950 2024-09-21T02:10:09.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.155+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef4151af0 con 0x7f5ef4162950 2024-09-21T02:10:09.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.159+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef40d3440 con 0x7f5ef4162950 2024-09-21T02:10:09.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.159+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5eec0be070 con 0x7f5ef4162950 2024-09-21T02:10:09.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.159+0000 7f5eea7fc640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ed8073200 0x7f5ed8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.159+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f5eec12e250 con 0x7f5ef4162950 2024-09-21T02:10:09.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.159+0000 7f5ef9dc7640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ed8073200 0x7f5ed8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.159+0000 7f5ef9dc7640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ed8073200 0x7f5ed8075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0005fd0 tx=0x7f5ef0005e80 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:09.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.163+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5eec016950 con 0x7f5ef4162950 2024-09-21T02:10:09.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.331+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ef4158330 con 0x7f5ef4162950 2024-09-21T02:10:09.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5eea7fc640 1 -- 172.21.15.96:0/3225173008 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f5eec0f8e70 con 0x7f5ef4162950 2024-09-21T02:10:09.339 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:09.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ed8073200 msgr2=0x7f5ed8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ed8073200 0x7f5ed8075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0005fd0 tx=0x7f5ef0005e80 comp rx=0 tx=0).stop 2024-09-21T02:10:09.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 msgr2=0x7f5ef40ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef40ce150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5eec01b410 tx=0x7f5eec0092c0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 shutdown_connections 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ed8073200 0x7f5ed8075710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef4162950 0x7f5ef40ce150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ef415f7d0 0x7f5ef414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 --2- 172.21.15.96:0/3225173008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ef40cf710 0x7f5ef4160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 >> 172.21.15.96:0/3225173008 conn(0x7f5ef40d1b10 msgr2=0x7f5ef40d3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 shutdown_connections 2024-09-21T02:10:09.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.335+0000 7f5efc02b640 1 -- 172.21.15.96:0/3225173008 wait complete. 2024-09-21T02:10:09.359 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-21T02:10:09.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:09.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:09.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:09.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:10:09.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:09.372 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:10:09.419 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:10:09.419 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:09.426 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:10:09.426 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:09.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:09.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 -- 172.21.15.96:0/4085573663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c44005920 msgr2=0x7f1c44005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 --2- 172.21.15.96:0/4085573663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c44005920 0x7f1c44005d30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0d1400 tx=0x7f1c5001d210 comp rx=0 tx=0).stop 2024-09-21T02:10:09.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 -- 172.21.15.96:0/4085573663 shutdown_connections 2024-09-21T02:10:09.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 --2- 172.21.15.96:0/4085573663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c44005920 0x7f1c44005d30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 -- 172.21.15.96:0/4085573663 >> 172.21.15.96:0/4085573663 conn(0x7f1c5c0d07a0 msgr2=0x7f1c5c0d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:09.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 -- 172.21.15.96:0/4085573663 shutdown_connections 2024-09-21T02:10:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.631+0000 7f1c61edc640 1 -- 172.21.15.96:0/4085573663 wait complete. 2024-09-21T02:10:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 Processor -- start 2024-09-21T02:10:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- start start 2024-09-21T02:10:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1c44005920 0x7f1c5c15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c15dc60 0x7f1c5c1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 0x7f1c5c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c15ed10 con 0x7f1c5c1599c0 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c15eb90 con 0x7f1c5c15dc60 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c15ee90 con 0x7f1c44005920 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 0x7f1c5c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 0x7f1c5c159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56792/0 (socket says 172.21.15.96:56792) 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 -- 172.21.15.96:0/2013941370 learned_addr learned my addr 172.21.15.96:0/2013941370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5b7fe640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1c44005920 0x7f1c5c15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5affd640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c15dc60 0x7f1c5c1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 -- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1c44005920 msgr2=0x7f1c5c15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1c44005920 0x7f1c5c15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 -- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c15dc60 msgr2=0x7f1c5c1f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c15dc60 0x7f1c5c1f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 -- 172.21.15.96:0/2013941370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c15a310 con 0x7f1c5c1599c0 2024-09-21T02:10:09.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c5bfff640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 0x7f1c5c159dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c15f5e0 tx=0x7f1c5400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:09.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54018070 con 0x7f1c5c1599c0 2024-09-21T02:10:09.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c54003980 con 0x7f1c5c1599c0 2024-09-21T02:10:09.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003dd0 con 0x7f1c5c1599c0 2024-09-21T02:10:09.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c15e070 con 0x7f1c5c1599c0 2024-09-21T02:10:09.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c1e64e0 con 0x7f1c5c1599c0 2024-09-21T02:10:09.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.635+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c15cc30 con 0x7f1c5c1599c0 2024-09-21T02:10:09.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.639+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1c5401b070 con 0x7f1c5c1599c0 2024-09-21T02:10:09.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.639+0000 7f1c43fff640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1c48073200 0x7f1c48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:09.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.639+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1c54093bb0 con 0x7f1c5c1599c0 2024-09-21T02:10:09.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.639+0000 7f1c5b7fe640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1c48073200 0x7f1c48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:09.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.639+0000 7f1c5b7fe640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1c48073200 0x7f1c48075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0036b0 tx=0x7f1c4c00c040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:09.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.643+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1c5c15a310 con 0x7f1c5c1599c0 2024-09-21T02:10:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.827+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1c5c1e6ec0 con 0x7f1c5c1599c0 2024-09-21T02:10:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.827+0000 7f1c43fff640 1 -- 172.21.15.96:0/2013941370 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f1c54097080 con 0x7f1c5c1599c0 2024-09-21T02:10:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1c48073200 msgr2=0x7f1c48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1c48073200 0x7f1c48075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0036b0 tx=0x7f1c4c00c040 comp rx=0 tx=0).stop 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 msgr2=0x7f1c5c159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 0x7f1c5c159dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c15f5e0 tx=0x7f1c5400dc10 comp rx=0 tx=0).stop 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 shutdown_connections 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1c48073200 0x7f1c48075710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c5c1599c0 0x7f1c5c159dd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c15dc60 0x7f1c5c1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 --2- 172.21.15.96:0/2013941370 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1c44005920 0x7f1c5c15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:09.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 >> 172.21.15.96:0/2013941370 conn(0x7f1c5c0d07a0 msgr2=0x7f1c5c06df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:09.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 shutdown_connections 2024-09-21T02:10:09.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.831+0000 7f1c61edc640 1 -- 172.21.15.96:0/2013941370 wait complete. 2024-09-21T02:10:09.855 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:10:09.855 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:09.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- 172.21.15.96:0/873156589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 msgr2=0x7f32040af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 --2- 172.21.15.96:0/873156589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f32040af4c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f32040b5d40 tx=0x7f31fc01d4e0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- 172.21.15.96:0/873156589 shutdown_connections 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 --2- 172.21.15.96:0/873156589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f32040af4c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 --2- 172.21.15.96:0/873156589 >> v2:172.21.15.122:3300/0 conn(0x7f32040cfed0 0x7f3204073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- 172.21.15.96:0/873156589 >> 172.21.15.96:0/873156589 conn(0x7f32040d13d0 msgr2=0x7f32040734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- 172.21.15.96:0/873156589 shutdown_connections 2024-09-21T02:10:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- 172.21.15.96:0/873156589 wait complete. 2024-09-21T02:10:10.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 Processor -- start 2024-09-21T02:10:10.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- start start 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f320414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32040cfac0 0x7f32041fb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32040cfed0 0x7f320414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f320406fa10 con 0x7f3204074140 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f320406f890 con 0x7f32040cfac0 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:09.999+0000 7f3209b72640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f320406fb90 con 0x7f32040cfed0 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f320414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f320414ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56804/0 (socket says 172.21.15.96:56804) 2024-09-21T02:10:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 -- 172.21.15.96:0/5273975 learned_addr learned my addr 172.21.15.96:0/5273975 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3203fff640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32040cfed0 0x7f320414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3202ffd640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32040cfac0 0x7f32041fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 -- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32040cfed0 msgr2=0x7f320414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32040cfed0 0x7f320414dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 -- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32040cfac0 msgr2=0x7f32041fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32040cfac0 0x7f32041fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 -- 172.21.15.96:0/5273975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3204151f80 con 0x7f3204074140 2024-09-21T02:10:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3202ffd640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32040cfac0 0x7f32041fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f32037fe640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f320414ed20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b440 tx=0x7f31f400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4015020 con 0x7f3204074140 2024-09-21T02:10:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31f4011980 con 0x7f3204074140 2024-09-21T02:10:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32040aead0 con 0x7f3204074140 2024-09-21T02:10:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4011dd0 con 0x7f3204074140 2024-09-21T02:10:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32040af090 con 0x7f3204074140 2024-09-21T02:10:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3204070110 con 0x7f3204074140 2024-09-21T02:10:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31f4018020 con 0x7f3204074140 2024-09-21T02:10:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f31ebfff640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073200 0x7f31f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f31f40944a0 con 0x7f3204074140 2024-09-21T02:10:10.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3202ffd640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073200 0x7f31f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.003+0000 7f3202ffd640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073200 0x7f31f0075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f320414a5a0 tx=0x7f31f8005cc0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.007+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f31f401f030 con 0x7f3204074140 2024-09-21T02:10:10.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32040b4c70 con 0x7f3204074140 2024-09-21T02:10:10.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f31ebfff640 1 -- 172.21.15.96:0/5273975 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f31f400d040 con 0x7f3204074140 2024-09-21T02:10:10.184 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:10.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073200 msgr2=0x7f31f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073200 0x7f31f0075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f320414a5a0 tx=0x7f31f8005cc0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 msgr2=0x7f320414ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f320414ed20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b440 tx=0x7f31f400b900 comp rx=0 tx=0).stop 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 shutdown_connections 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f31f0073200 0x7f31f0075710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32040cfed0 0x7f320414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32040cfac0 0x7f32041fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 --2- 172.21.15.96:0/5273975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3204074140 0x7f320414ed20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 >> 172.21.15.96:0/5273975 conn(0x7f32040d13d0 msgr2=0x7f3204150850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 shutdown_connections 2024-09-21T02:10:10.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.179+0000 7f3209b72640 1 -- 172.21.15.96:0/5273975 wait complete. 2024-09-21T02:10:10.205 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi122.front.sepia.ceph.com... 2024-09-21T02:10:10.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:10.206 DEBUG:teuthology.orchestra.run.smithi122:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:10:10.224 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:10.224 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:10:10.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:10.225 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:10:10.277 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:10:10.278 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:10.284 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:10:10.284 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:10.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3819095311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80cec00 msgr2=0x7fc9d80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3819095311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80cec00 0x7fc9d80cf010 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc9d806a1c0 tx=0x7fc9d001c770 comp rx=0 tx=0).stop 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3819095311 shutdown_connections 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3819095311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80cec00 0x7fc9d80cf010 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3819095311 >> v2:172.21.15.122:3300/0 conn(0x7fc9d806e2c0 0x7fc9d80c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3819095311 >> 172.21.15.96:0/3819095311 conn(0x7fc9d806eea0 msgr2=0x7fc9d806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3819095311 shutdown_connections 2024-09-21T02:10:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3819095311 wait complete. 2024-09-21T02:10:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 Processor -- start 2024-09-21T02:10:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.523+0000 7fc9dfd5d640 1 -- start start 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d806e2c0 0x7fc9d814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 0x7fc9d814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc9d80cec00 0x7fc9d81f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8169c60 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8169ae0 con 0x7fc9d806e2c0 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d8169de0 con 0x7fc9d80cec00 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 0x7fc9d814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 0x7fc9d814cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56822/0 (socket says 172.21.15.96:56822) 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 -- 172.21.15.96:0/3543427660 learned_addr learned my addr 172.21.15.96:0/3543427660 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9ddaf9640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d806e2c0 0x7fc9d814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9de2fa640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc9d80cec00 0x7fc9d81f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 -- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc9d80cec00 msgr2=0x7fc9d81f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc9d80cec00 0x7fc9d81f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 -- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d806e2c0 msgr2=0x7fc9d814c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d806e2c0 0x7fc9d814c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 -- 172.21.15.96:0/3543427660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d81583f0 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9de2fa640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc9d80cec00 0x7fc9d81f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dd2f8640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 0x7fc9d814cd30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400cb70 tx=0x7fc9d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4016020 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9d400ab60 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d814dfc0 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4012420 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d814e580 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d8073310 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.527+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc9d4019020 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.531+0000 7fc9ca7fc640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc9bc073200 0x7fc9bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.531+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc9d40949d0 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.531+0000 7fc9ddaf9640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc9bc073200 0x7fc9bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.531+0000 7fc9ddaf9640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc9bc073200 0x7fc9bc075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0058d0 tx=0x7fc9cc005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.535+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc9d4020020 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9d815b890 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9ca7fc640 1 -- 172.21.15.96:0/3543427660 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fc9d405f570 con 0x7fc9d80c4ac0 2024-09-21T02:10:10.724 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:10.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc9bc073200 msgr2=0x7fc9bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc9bc073200 0x7fc9bc075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0058d0 tx=0x7fc9cc005860 comp rx=0 tx=0).stop 2024-09-21T02:10:10.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 msgr2=0x7fc9d814cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 0x7fc9d814cd30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400cb70 tx=0x7fc9d400cc60 comp rx=0 tx=0).stop 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 shutdown_connections 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fc9bc073200 0x7fc9bc075710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc9d80cec00 0x7fc9d81f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d80c4ac0 0x7fc9d814cd30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 --2- 172.21.15.96:0/3543427660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d806e2c0 0x7fc9d814c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 >> 172.21.15.96:0/3543427660 conn(0x7fc9d806eea0 msgr2=0x7fc9d814ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 shutdown_connections 2024-09-21T02:10:10.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.719+0000 7fc9dfd5d640 1 -- 172.21.15.96:0/3543427660 wait complete. 2024-09-21T02:10:10.744 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:10:10.745 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:10.750 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b46f6a640 1 -- 172.21.15.96:0/729984025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b28004d50 msgr2=0x7f3b28005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/729984025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b28004d50 0x7f3b28005160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c610 tx=0x7f3b3801fa40 comp rx=0 tx=0).stop 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b45507640 1 -- 172.21.15.96:0/729984025 reap_dead start 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b46f6a640 1 -- 172.21.15.96:0/729984025 shutdown_connections 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b46f6a640 1 -- 172.21.15.96:0/729984025 >> 172.21.15.96:0/729984025 conn(0x7f3b4006eea0 msgr2=0x7f3b4006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b46f6a640 1 -- 172.21.15.96:0/729984025 shutdown_connections 2024-09-21T02:10:10.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.943+0000 7f3b46f6a640 1 -- 172.21.15.96:0/729984025 wait complete. 2024-09-21T02:10:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 Processor -- start 2024-09-21T02:10:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 -- start start 2024-09-21T02:10:10.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b28004d50 0x7f3b4014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 0x7f3b400cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3b400cb860 0x7f3b401f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400c5410 con 0x7f3b40153620 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4006e7e0 con 0x7f3b28004d50 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b46f6a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400c5590 con 0x7f3b400cb860 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 0x7f3b400cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 0x7f3b400cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56838/0 (socket says 172.21.15.96:56838) 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 -- 172.21.15.96:0/4023783619 learned_addr learned my addr 172.21.15.96:0/4023783619 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b44d06640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b28004d50 0x7f3b4014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b45507640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3b400cb860 0x7f3b401f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 -- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3b400cb860 msgr2=0x7f3b401f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3b400cb860 0x7f3b401f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 -- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b28004d50 msgr2=0x7f3b4014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b28004d50 0x7f3b4014d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 -- 172.21.15.96:0/4023783619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b400cbc70 con 0x7f3b40153620 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b45507640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3b400cb860 0x7f3b401f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b44d06640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b28004d50 0x7f3b4014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3ffff640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 0x7f3b400cb320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b300109e0 tx=0x7f3b30010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.947+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000ecb0 con 0x7f3b40153620 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b3000ee50 con 0x7f3b40153620 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b400c96f0 con 0x7f3b40153620 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004270 con 0x7f3b40153620 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b400c9cb0 con 0x7f3b40153620 2024-09-21T02:10:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b400c9460 con 0x7f3b40153620 2024-09-21T02:10:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3b30003600 con 0x7f3b40153620 2024-09-21T02:10:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b3d7fa640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3b2c073230 0x7f3b2c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3b300957a0 con 0x7f3b40153620 2024-09-21T02:10:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b44d06640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3b2c073230 0x7f3b2c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.951+0000 7f3b44d06640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3b2c073230 0x7f3b2c075740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009b70 tx=0x7f3b340091e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:10.955+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3b400cbc70 con 0x7f3b40153620 2024-09-21T02:10:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.139+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b400c4790 con 0x7f3b40153620 2024-09-21T02:10:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.139+0000 7f3b3d7fa640 1 -- 172.21.15.96:0/4023783619 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f3b300604c0 con 0x7f3b40153620 2024-09-21T02:10:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3b2c073230 msgr2=0x7f3b2c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3b2c073230 0x7f3b2c075740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009b70 tx=0x7f3b340091e0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 msgr2=0x7f3b400cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 0x7f3b400cb320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b300109e0 tx=0x7f3b30010ea0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 shutdown_connections 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3b2c073230 0x7f3b2c075740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3b400cb860 0x7f3b401f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b40153620 0x7f3b400cb320 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 --2- 172.21.15.96:0/4023783619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b28004d50 0x7f3b4014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 >> 172.21.15.96:0/4023783619 conn(0x7f3b4006eea0 msgr2=0x7f3b400b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 shutdown_connections 2024-09-21T02:10:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.143+0000 7f3b46f6a640 1 -- 172.21.15.96:0/4023783619 wait complete. 2024-09-21T02:10:11.168 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-21T02:10:11.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:11.168 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-21T02:10:11.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:11.178 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:10:11.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:11.179 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T02:10:11.227 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:10:11.227 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:11.233 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:10:11.233 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:11.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:11.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 -- 172.21.15.96:0/456414583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 msgr2=0x7f10540cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 --2- 172.21.15.96:0/456414583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f10540cff20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f105406f990 tx=0x7f104801c890 comp rx=0 tx=0).stop 2024-09-21T02:10:11.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 -- 172.21.15.96:0/456414583 shutdown_connections 2024-09-21T02:10:11.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 --2- 172.21.15.96:0/456414583 >> v2:172.21.15.122:3300/0 conn(0x7f105415fb60 0x7f10540d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 --2- 172.21.15.96:0/456414583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f10540cff20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 --2- 172.21.15.96:0/456414583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f105415b310 0x7f10540cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 -- 172.21.15.96:0/456414583 >> 172.21.15.96:0/456414583 conn(0x7f105406fbe0 msgr2=0x7f105406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:11.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 -- 172.21.15.96:0/456414583 shutdown_connections 2024-09-21T02:10:11.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.431+0000 7f10598f2640 1 -- 172.21.15.96:0/456414583 wait complete. 2024-09-21T02:10:11.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 Processor -- start 2024-09-21T02:10:11.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 -- start start 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f105415b310 0x7f1054152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f1054156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f105415fb60 0x7f10541f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f105406e3e0 con 0x7f105415f750 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f105406e260 con 0x7f105415fb60 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10598f2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f105406e560 con 0x7f105415b310 2024-09-21T02:10:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10527fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f1054156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10527fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f1054156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56866/0 (socket says 172.21.15.96:56866) 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10527fc640 1 -- 172.21.15.96:0/3287832024 learned_addr learned my addr 172.21.15.96:0/3287832024 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f1052ffd640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f105415b310 0x7f1054152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.435+0000 7f10537fe640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f105415fb60 0x7f10541f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10527fc640 1 -- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f105415b310 msgr2=0x7f1054152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10527fc640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f105415b310 0x7f1054152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10527fc640 1 -- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f105415fb60 msgr2=0x7f10541f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10527fc640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f105415fb60 0x7f10541f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10527fc640 1 -- 172.21.15.96:0/3287832024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10541558e0 con 0x7f105415f750 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10527fc640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f1054156980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10480b0040 tx=0x7f1048004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10480b2040 con 0x7f105415f750 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1048005030 con 0x7f105415f750 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1054155bc0 con 0x7f105415f750 2024-09-21T02:10:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048005460 con 0x7f105415f750 2024-09-21T02:10:11.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1054156180 con 0x7f105415f750 2024-09-21T02:10:11.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10540cebc0 con 0x7f105415f750 2024-09-21T02:10:11.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f10480046d0 con 0x7f105415f750 2024-09-21T02:10:11.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f103bfff640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1040073200 0x7f1040075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f104812ddb0 con 0x7f105415f750 2024-09-21T02:10:11.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.439+0000 7f1052ffd640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1040073200 0x7f1040075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.443+0000 7f1052ffd640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1040073200 0x7f1040075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1044004630 tx=0x7f104400b260 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:11.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.443+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f10480bc060 con 0x7f105415f750 2024-09-21T02:10:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.619+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1054155650 con 0x7f105415f750 2024-09-21T02:10:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.623+0000 7f103bfff640 1 -- 172.21.15.96:0/3287832024 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f10480164b0 con 0x7f105415f750 2024-09-21T02:10:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1040073200 msgr2=0x7f1040075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1040073200 0x7f1040075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1044004630 tx=0x7f104400b260 comp rx=0 tx=0).stop 2024-09-21T02:10:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 msgr2=0x7f1054156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f1054156980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10480b0040 tx=0x7f1048004040 comp rx=0 tx=0).stop 2024-09-21T02:10:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 shutdown_connections 2024-09-21T02:10:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1040073200 0x7f1040075710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f105415fb60 0x7f10541f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f105415f750 0x7f1054156980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 --2- 172.21.15.96:0/3287832024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f105415b310 0x7f1054152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 >> 172.21.15.96:0/3287832024 conn(0x7f105406fbe0 msgr2=0x7f1054165730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 shutdown_connections 2024-09-21T02:10:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.627+0000 7f10598f2640 1 -- 172.21.15.96:0/3287832024 wait complete. 2024-09-21T02:10:11.650 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:10:11.650 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:10:11.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 -- 172.21.15.96:0/4088607936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa8806e5b0 msgr2=0x7faa880cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 --2- 172.21.15.96:0/4088607936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa8806e5b0 0x7faa880cab60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa8806a1c0 tx=0x7faa8001cb40 comp rx=0 tx=0).stop 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 -- 172.21.15.96:0/4088607936 shutdown_connections 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 --2- 172.21.15.96:0/4088607936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa8806e5b0 0x7faa880cab60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 --2- 172.21.15.96:0/4088607936 >> v2:172.21.15.122:3300/0 conn(0x7faa88155240 0x7faa88073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 -- 172.21.15.96:0/4088607936 >> 172.21.15.96:0/4088607936 conn(0x7faa8806eea0 msgr2=0x7faa8806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 -- 172.21.15.96:0/4088607936 shutdown_connections 2024-09-21T02:10:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 -- 172.21.15.96:0/4088607936 wait complete. 2024-09-21T02:10:11.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 Processor -- start 2024-09-21T02:10:11.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.811+0000 7faa8d412640 1 -- start start 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 0x7faa881545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa88155240 0x7faa880a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa880aef10 0x7faa880aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa880cbd80 con 0x7faa88073970 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa880cbc00 con 0x7faa88155240 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faa880cbf00 con 0x7faa880aef10 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 0x7faa881545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 0x7faa881545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56894/0 (socket says 172.21.15.96:56894) 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 -- 172.21.15.96:0/3743463277 learned_addr learned my addr 172.21.15.96:0/3743463277 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 -- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa880aef10 msgr2=0x7faa880aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa880aef10 0x7faa880aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 -- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa88155240 msgr2=0x7faa880a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa867fc640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa88155240 0x7faa880a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa88155240 0x7faa880a9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 -- 172.21.15.96:0/3743463277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa88154b30 con 0x7faa88073970 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa86ffd640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 0x7faa881545f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faa7800d950 tx=0x7faa7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78017070 con 0x7faa88073970 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa78009d20 con 0x7faa88073970 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa880cf320 con 0x7faa88073970 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa780106c0 con 0x7faa88073970 2024-09-21T02:10:11.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa880c7fb0 con 0x7faa88073970 2024-09-21T02:10:11.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa880cab60 con 0x7faa88073970 2024-09-21T02:10:11.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.815+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faa7801a070 con 0x7faa88073970 2024-09-21T02:10:11.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.819+0000 7faa6f7fe640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faa74073230 0x7faa74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:11.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.819+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7faa78094780 con 0x7faa88073970 2024-09-21T02:10:11.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.819+0000 7faa867fc640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faa74073230 0x7faa74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:11.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.819+0000 7faa867fc640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faa74073230 0x7faa74075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa880cfd10 tx=0x7faa7c009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:11.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:11.823+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faa7805f430 con 0x7faa88073970 2024-09-21T02:10:12.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faa8806e9d0 con 0x7faa88073970 2024-09-21T02:10:12.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa6f7fe640 1 -- 172.21.15.96:0/3743463277 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7faa7805ed30 con 0x7faa88073970 2024-09-21T02:10:12.012 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-21T02:10:12.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faa74073230 msgr2=0x7faa74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faa74073230 0x7faa74075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa880cfd10 tx=0x7faa7c009040 comp rx=0 tx=0).stop 2024-09-21T02:10:12.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 msgr2=0x7faa881545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 0x7faa881545f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faa7800d950 tx=0x7faa7800de10 comp rx=0 tx=0).stop 2024-09-21T02:10:12.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 shutdown_connections 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faa74073230 0x7faa74075740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa880aef10 0x7faa880aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa88155240 0x7faa880a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 --2- 172.21.15.96:0/3743463277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa88073970 0x7faa881545f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 >> 172.21.15.96:0/3743463277 conn(0x7faa8806eea0 msgr2=0x7faa880c5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 shutdown_connections 2024-09-21T02:10:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.007+0000 7faa8d412640 1 -- 172.21.15.96:0/3743463277 wait complete. 2024-09-21T02:10:12.034 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi122.front.sepia.ceph.com... 2024-09-21T02:10:12.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:12.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-21T02:10:12.048 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:12.048 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:10:12.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:12.048 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T02:10:12.095 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:10:12.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 -- 172.21.15.96:0/558949320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 msgr2=0x7ff7400d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 --2- 172.21.15.96:0/558949320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff7400d2460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7400d42c0 tx=0x7ff73001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 -- 172.21.15.96:0/558949320 shutdown_connections 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 --2- 172.21.15.96:0/558949320 >> v2:172.21.15.122:3300/0 conn(0x7ff7400d29a0 0x7ff7400d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 --2- 172.21.15.96:0/558949320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff7400d2460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 -- 172.21.15.96:0/558949320 >> 172.21.15.96:0/558949320 conn(0x7ff7400d1b10 msgr2=0x7ff7400d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 -- 172.21.15.96:0/558949320 shutdown_connections 2024-09-21T02:10:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.247+0000 7ff747ec5640 1 -- 172.21.15.96:0/558949320 wait complete. 2024-09-21T02:10:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 Processor -- start 2024-09-21T02:10:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 -- start start 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7400cf710 0x7ff7400b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7400d29a0 0x7ff7400ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff74014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff74016bfc0 con 0x7ff74015f9a0 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff74016c140 con 0x7ff7400d29a0 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff74016c2c0 con 0x7ff7400cf710 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff74014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff74014b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56926/0 (socket says 172.21.15.96:56926) 2024-09-21T02:10:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 -- 172.21.15.96:0/1637410678 learned_addr learned my addr 172.21.15.96:0/1637410678 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff745c61640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7400cf710 0x7ff7400b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff745460640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7400d29a0 0x7ff7400ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 -- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7400cf710 msgr2=0x7ff7400b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7400cf710 0x7ff7400b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 -- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7400d29a0 msgr2=0x7ff7400ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7400d29a0 0x7ff7400ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 -- 172.21.15.96:0/1637410678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400ce6c0 con 0x7ff74015f9a0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff746462640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff74014b2d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00cab0 tx=0x7ff73c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c007d90 con 0x7ff74015f9a0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff73c003a90 con 0x7ff74015f9a0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74014d810 con 0x7ff74015f9a0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c004020 con 0x7ff74015f9a0 2024-09-21T02:10:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.251+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401544f0 con 0x7ff74015f9a0 2024-09-21T02:10:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.255+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74014d580 con 0x7ff74015f9a0 2024-09-21T02:10:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.255+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff73c0041c0 con 0x7ff74015f9a0 2024-09-21T02:10:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.255+0000 7ff7367fc640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff724073200 0x7ff724075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.255+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff73c095060 con 0x7ff74015f9a0 2024-09-21T02:10:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.255+0000 7ff745c61640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff724073200 0x7ff724075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.255+0000 7ff745c61640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff724073200 0x7ff724075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7400b7610 tx=0x7ff738008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:12.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.259+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff7400ce6c0 con 0x7ff74015f9a0 2024-09-21T02:10:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.467+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7400d2db0 con 0x7ff74015f9a0 2024-09-21T02:10:12.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.467+0000 7ff7367fc640 1 -- 172.21.15.96:0/1637410678 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7ff73c014070 con 0x7ff74015f9a0 2024-09-21T02:10:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:12.474 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:10:12.474 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:10:12.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff724073200 msgr2=0x7ff724075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff724073200 0x7ff724075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7400b7610 tx=0x7ff738008040 comp rx=0 tx=0).stop 2024-09-21T02:10:12.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 msgr2=0x7ff74014b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff74014b2d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00cab0 tx=0x7ff73c00cf70 comp rx=0 tx=0).stop 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 shutdown_connections 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff724073200 0x7ff724075710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74015f9a0 0x7ff74014b2d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7400d29a0 0x7ff7400ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 --2- 172.21.15.96:0/1637410678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7400cf710 0x7ff7400b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 >> 172.21.15.96:0/1637410678 conn(0x7ff7400d1b10 msgr2=0x7ff74014f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 shutdown_connections 2024-09-21T02:10:12.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.471+0000 7ff747ec5640 1 -- 172.21.15.96:0/1637410678 wait complete. 2024-09-21T02:10:12.496 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:10:12.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 -- 172.21.15.96:0/3493439275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 msgr2=0x7f6c580b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 --2- 172.21.15.96:0/3493439275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c580b4100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c5815dcc0 tx=0x7f6c5001c9a0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 -- 172.21.15.96:0/3493439275 shutdown_connections 2024-09-21T02:10:12.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 --2- 172.21.15.96:0/3493439275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c580b4100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 -- 172.21.15.96:0/3493439275 >> 172.21.15.96:0/3493439275 conn(0x7f6c580cf110 msgr2=0x7f6c580cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:12.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 -- 172.21.15.96:0/3493439275 shutdown_connections 2024-09-21T02:10:12.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.643+0000 7f6c60481640 1 -- 172.21.15.96:0/3493439275 wait complete. 2024-09-21T02:10:12.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 Processor -- start 2024-09-21T02:10:12.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- start start 2024-09-21T02:10:12.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c580d0160 0x7f6c581fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c58156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c5815cba0 0x7f6c58168c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581539f0 con 0x7f6c580d0570 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58153870 con 0x7f6c5815cba0 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5816b340 con 0x7f6c580d0160 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c58156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c58156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56946/0 (socket says 172.21.15.96:56946) 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 -- 172.21.15.96:0/3314398010 learned_addr learned my addr 172.21.15.96:0/3314398010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5ea1e640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c5815cba0 0x7f6c58168c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 -- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c580d0160 msgr2=0x7f6c581fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5e21d640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c580d0160 0x7f6c581fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c580d0160 0x7f6c581fcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 -- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c5815cba0 msgr2=0x7f6c58168c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c5815cba0 0x7f6c58168c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 -- 172.21.15.96:0/3314398010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c58169150 con 0x7f6c580d0570 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5ea1e640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c5815cba0 0x7f6c58168c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c5da1c640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c58156b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c3f0 tx=0x7f6c4800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48017070 con 0x7f6c580d0570 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c480053c0 con 0x7f6c580d0570 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c580cc2f0 con 0x7f6c580d0570 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48003860 con 0x7f6c580d0570 2024-09-21T02:10:12.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c581fc0b0 con 0x7f6c580d0570 2024-09-21T02:10:12.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.647+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c580b4100 con 0x7f6c580d0570 2024-09-21T02:10:12.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.651+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c4801a070 con 0x7f6c580d0570 2024-09-21T02:10:12.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.651+0000 7f6c4effd640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6c3c073200 0x7f6c3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:12.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.651+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6c48093a10 con 0x7f6c580d0570 2024-09-21T02:10:12.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.651+0000 7f6c5e21d640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6c3c073200 0x7f6c3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:12.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.651+0000 7f6c5e21d640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6c3c073200 0x7f6c3c075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6c54001230 tx=0x7f6c54009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:12.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.655+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6c58169150 con 0x7f6c580d0570 2024-09-21T02:10:12.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.859+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c5806a170 con 0x7f6c580d0570 2024-09-21T02:10:12.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c4effd640 1 -- 172.21.15.96:0/3314398010 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f6c4805e630 con 0x7f6c580d0570 2024-09-21T02:10:12.867 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:12.867 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:10:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:10:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6c3c073200 msgr2=0x7f6c3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6c3c073200 0x7f6c3c075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6c54001230 tx=0x7f6c54009040 comp rx=0 tx=0).stop 2024-09-21T02:10:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 msgr2=0x7f6c58156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c58156b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c3f0 tx=0x7f6c4800c8b0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 shutdown_connections 2024-09-21T02:10:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6c3c073200 0x7f6c3c075710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c5815cba0 0x7f6c58168c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c580d0570 0x7f6c58156b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 --2- 172.21.15.96:0/3314398010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c580d0160 0x7f6c581fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 >> 172.21.15.96:0/3314398010 conn(0x7f6c580cf110 msgr2=0x7f6c580c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 shutdown_connections 2024-09-21T02:10:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:12.863+0000 7f6c60481640 1 -- 172.21.15.96:0/3314398010 wait complete. 2024-09-21T02:10:12.888 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 -- 172.21.15.96:0/2730966303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4155240 msgr2=0x7f9dc4073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 --2- 172.21.15.96:0/2730966303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4155240 0x7f9dc4073430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9dc406a1c0 tx=0x7f9db401c670 comp rx=0 tx=0).stop 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 -- 172.21.15.96:0/2730966303 shutdown_connections 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 --2- 172.21.15.96:0/2730966303 >> v2:172.21.15.122:3300/0 conn(0x7f9dc4073970 0x7f9dc406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 --2- 172.21.15.96:0/2730966303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4155240 0x7f9dc4073430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 -- 172.21.15.96:0/2730966303 >> 172.21.15.96:0/2730966303 conn(0x7f9dc406eea0 msgr2=0x7f9dc406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 -- 172.21.15.96:0/2730966303 shutdown_connections 2024-09-21T02:10:13.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.031+0000 7f9dca948640 1 -- 172.21.15.96:0/2730966303 wait complete. 2024-09-21T02:10:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 Processor -- start 2024-09-21T02:10:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 -- start start 2024-09-21T02:10:13.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9dc406e3e0 0x7f9dc414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 0x7f9dc41572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc4155240 0x7f9dc41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4165ef0 con 0x7f9dc4073970 2024-09-21T02:10:13.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4165d70 con 0x7f9dc4155240 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc4166070 con 0x7f9dc406e3e0 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 0x7f9dc41572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 0x7f9dc41572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56966/0 (socket says 172.21.15.96:56966) 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 -- 172.21.15.96:0/4008525563 learned_addr learned my addr 172.21.15.96:0/4008525563 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 -- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9dc406e3e0 msgr2=0x7f9dc414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9dc406e3e0 0x7f9dc414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 -- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc4155240 msgr2=0x7f9dc41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dc8ee5640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc4155240 0x7f9dc41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc4155240 0x7f9dc41577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 -- 172.21.15.96:0/4008525563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc414f950 con 0x7f9dc4073970 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dbbfff640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 0x7f9dc41572b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00d750 tx=0x7f9dac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac017070 con 0x7f9dc4073970 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dac009d20 con 0x7f9dc4073970 2024-09-21T02:10:13.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc40b5e30 con 0x7f9dc4073970 2024-09-21T02:10:13.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac005280 con 0x7f9dc4073970 2024-09-21T02:10:13.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.035+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc40b5b50 con 0x7f9dc4073970 2024-09-21T02:10:13.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.039+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc406dea0 con 0x7f9dc4073970 2024-09-21T02:10:13.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.039+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9dac005420 con 0x7f9dc4073970 2024-09-21T02:10:13.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.039+0000 7f9dc17fa640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9da4073200 0x7f9da4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.039+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9dac0946a0 con 0x7f9dc4073970 2024-09-21T02:10:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.039+0000 7f9dc3fff640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9da4073200 0x7f9da4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.039+0000 7f9dc3fff640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9da4073200 0x7f9da4075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9db40af000 tx=0x7f9db4019410 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.043+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9dc414f950 con 0x7f9dc4073970 2024-09-21T02:10:13.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.203+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dc406e9d0 con 0x7f9dc4073970 2024-09-21T02:10:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dc17fa640 1 -- 172.21.15.96:0/4008525563 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18985 (secure 0 0 0) 0x7f9dac00de60 con 0x7f9dc4073970 2024-09-21T02:10:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:13.211 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":28,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:51.588360+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:10:13.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9da4073200 msgr2=0x7f9da4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9da4073200 0x7f9da4075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9db40af000 tx=0x7f9db4019410 comp rx=0 tx=0).stop 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 msgr2=0x7f9dc41572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 0x7f9dc41572b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00d750 tx=0x7f9dac00dc10 comp rx=0 tx=0).stop 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 shutdown_connections 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9da4073200 0x7f9da4075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc4155240 0x7f9dc41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dc4073970 0x7f9dc41572b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 --2- 172.21.15.96:0/4008525563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9dc406e3e0 0x7f9dc414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 >> 172.21.15.96:0/4008525563 conn(0x7f9dc406eea0 msgr2=0x7f9dc415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 shutdown_connections 2024-09-21T02:10:13.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.207+0000 7f9dca948640 1 -- 172.21.15.96:0/4008525563 wait complete. 2024-09-21T02:10:13.232 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:10:13.232 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:10:13.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 -- 172.21.15.96:0/890773901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfaa0 msgr2=0x7fee3815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 --2- 172.21.15.96:0/890773901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfaa0 0x7fee3815ef20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fee38071f70 tx=0x7fee3001cc30 comp rx=0 tx=0).stop 2024-09-21T02:10:13.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 -- 172.21.15.96:0/890773901 shutdown_connections 2024-09-21T02:10:13.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 --2- 172.21.15.96:0/890773901 >> v2:172.21.15.122:3300/0 conn(0x7fee38074310 0x7fee380af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 --2- 172.21.15.96:0/890773901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfaa0 0x7fee3815ef20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 -- 172.21.15.96:0/890773901 >> 172.21.15.96:0/890773901 conn(0x7fee380d13d0 msgr2=0x7fee380734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 -- 172.21.15.96:0/890773901 shutdown_connections 2024-09-21T02:10:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.375+0000 7fee3ccad640 1 -- 172.21.15.96:0/890773901 wait complete. 2024-09-21T02:10:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 Processor -- start 2024-09-21T02:10:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 -- start start 2024-09-21T02:10:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fee380cfaa0 0x7fee38148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 0x7fee380b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee38147d90 0x7fee380b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee3806fcf0 con 0x7fee380cfeb0 2024-09-21T02:10:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fee3806fb70 con 0x7fee38147d90 2024-09-21T02:10:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fee3806fe70 con 0x7fee380cfaa0 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 0x7fee380b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 0x7fee380b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56996/0 (socket says 172.21.15.96:56996) 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 -- 172.21.15.96:0/2579591010 learned_addr learned my addr 172.21.15.96:0/2579591010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3659c640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fee380cfaa0 0x7fee38148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee36d9d640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee38147d90 0x7fee380b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 -- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fee380cfaa0 msgr2=0x7fee38148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fee380cfaa0 0x7fee38148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 -- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee38147d90 msgr2=0x7fee380b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee38147d90 0x7fee380b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 -- 172.21.15.96:0/2579591010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee38132cd0 con 0x7fee380cfeb0 2024-09-21T02:10:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee35d9b640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 0x7fee380b38b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fee2000b410 tx=0x7fee2000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20015030 con 0x7fee380cfeb0 2024-09-21T02:10:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee20004020 con 0x7fee380cfeb0 2024-09-21T02:10:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200044b0 con 0x7fee380cfeb0 2024-09-21T02:10:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee381491b0 con 0x7fee380cfeb0 2024-09-21T02:10:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.379+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381334b0 con 0x7fee380cfeb0 2024-09-21T02:10:13.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.383+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee38132f40 con 0x7fee380cfeb0 2024-09-21T02:10:13.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.383+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fee20018070 con 0x7fee380cfeb0 2024-09-21T02:10:13.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.383+0000 7fee2effd640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fee18073230 0x7fee18075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.383+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fee20094180 con 0x7fee380cfeb0 2024-09-21T02:10:13.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.383+0000 7fee3659c640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fee18073230 0x7fee18075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.383+0000 7fee3659c640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fee18073230 0x7fee18075740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee300af000 tx=0x7fee3001b470 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:13.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.387+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fee2005ee20 con 0x7fee380cfeb0 2024-09-21T02:10:13.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.579+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee380af4c0 con 0x7fee380cfeb0 2024-09-21T02:10:13.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.579+0000 7fee2effd640 1 -- 172.21.15.96:0/2579591010 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7fee2005e720 con 0x7fee380cfeb0 2024-09-21T02:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:10:13.586 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fee18073230 msgr2=0x7fee18075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fee18073230 0x7fee18075740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee300af000 tx=0x7fee3001b470 comp rx=0 tx=0).stop 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 msgr2=0x7fee380b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 0x7fee380b38b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fee2000b410 tx=0x7fee2000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 shutdown_connections 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fee18073230 0x7fee18075740 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee38147d90 0x7fee380b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee380cfeb0 0x7fee380b38b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 --2- 172.21.15.96:0/2579591010 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fee380cfaa0 0x7fee38148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 >> 172.21.15.96:0/2579591010 conn(0x7fee380d13d0 msgr2=0x7fee381522d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:13.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 shutdown_connections 2024-09-21T02:10:13.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.583+0000 7fee3ccad640 1 -- 172.21.15.96:0/2579591010 wait complete. 2024-09-21T02:10:13.606 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:10:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 -- 172.21.15.96:0/2288729071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c40d0630 msgr2=0x7f54c40cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 --2- 172.21.15.96:0/2288729071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c40d0630 0x7f54c40cc480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54c40737e0 tx=0x7f54bc01cb50 comp rx=0 tx=0).stop 2024-09-21T02:10:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 -- 172.21.15.96:0/2288729071 shutdown_connections 2024-09-21T02:10:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 --2- 172.21.15.96:0/2288729071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c40d0630 0x7f54c40cc480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 --2- 172.21.15.96:0/2288729071 >> v2:172.21.15.122:3300/0 conn(0x7f54c415b310 0x7f54c40cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 -- 172.21.15.96:0/2288729071 >> 172.21.15.96:0/2288729071 conn(0x7f54c406fbe0 msgr2=0x7f54c406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:13.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 -- 172.21.15.96:0/2288729071 shutdown_connections 2024-09-21T02:10:13.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.751+0000 7f54c8c84640 1 -- 172.21.15.96:0/2288729071 wait complete. 2024-09-21T02:10:13.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 Processor -- start 2024-09-21T02:10:13.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 -- start start 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c415b310 0x7f54c4165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 0x7f54c4159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c4153df0 0x7f54c41598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4168d90 con 0x7f54c415f750 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4168c10 con 0x7f54c415b310 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f54c4168f10 con 0x7f54c4153df0 2024-09-21T02:10:13.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 0x7f54c4159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 0x7f54c4159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57002/0 (socket says 172.21.15.96:57002) 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 -- 172.21.15.96:0/3910260259 learned_addr learned my addr 172.21.15.96:0/3910260259 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c259c640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c415b310 0x7f54c4165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c2d9d640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c4153df0 0x7f54c41598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 -- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c4153df0 msgr2=0x7f54c41598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c4153df0 0x7f54c41598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 -- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c415b310 msgr2=0x7f54c4165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c415b310 0x7f54c4165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 -- 172.21.15.96:0/3910260259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c41422e0 con 0x7f54c415f750 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c259c640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c415b310 0x7f54c4165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c1d9b640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 0x7f54c4159390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c3c0 tx=0x7f54b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x7f54c415f750 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54b8003600 con 0x7f54c415f750 2024-09-21T02:10:13.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c41425c0 con 0x7f54c415f750 2024-09-21T02:10:13.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8003a50 con 0x7f54c415f750 2024-09-21T02:10:13.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.755+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c4142b80 con 0x7f54c415f750 2024-09-21T02:10:13.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.759+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c4142050 con 0x7f54c415f750 2024-09-21T02:10:13.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.759+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54b801a070 con 0x7f54c415f750 2024-09-21T02:10:13.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.759+0000 7f54b2ffd640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54a4073200 0x7f54a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:13.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.759+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f54b8093870 con 0x7f54c415f750 2024-09-21T02:10:13.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.759+0000 7f54c259c640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54a4073200 0x7f54a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:13.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.759+0000 7f54c259c640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54a4073200 0x7f54a4075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005fd0 tx=0x7f54b4005d00 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:13.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.763+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54b805e4a0 con 0x7f54c415f750 2024-09-21T02:10:13.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.971+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54c40cc480 con 0x7f54c415f750 2024-09-21T02:10:13.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.971+0000 7f54b2ffd640 1 -- 172.21.15.96:0/3910260259 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f54b805dda0 con 0x7f54c415f750 2024-09-21T02:10:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:13.978 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-21T02:09:32:586826+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2499264121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2499264121},{"type":"v1","addr":"172.21.15.122:6835","nonce":2499264121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/4051604014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":4051604014},{"type":"v1","addr":"172.21.15.122:6837","nonce":4051604014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1304686720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1304686720},{"type":"v1","addr":"172.21.15.96:6833","nonce":1304686720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:09:10.893814+0000","modified":"2024-09-21T02:09:31.976954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4353},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4353":{"gid":4353,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2262930956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2262930956},{"type":"v1","addr":"172.21.15.96:6835","nonce":2262930956}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4353,"qdb_cluster":[4353]},"id":1}]} 2024-09-21T02:10:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-21T02:10:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54a4073200 msgr2=0x7f54a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54a4073200 0x7f54a4075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005fd0 tx=0x7f54b4005d00 comp rx=0 tx=0).stop 2024-09-21T02:10:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 msgr2=0x7f54c4159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 0x7f54c4159390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c3c0 tx=0x7f54b800c880 comp rx=0 tx=0).stop 2024-09-21T02:10:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 shutdown_connections 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f54a4073200 0x7f54a4075710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54c4153df0 0x7f54c41598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54c415f750 0x7f54c4159390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 --2- 172.21.15.96:0/3910260259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54c415b310 0x7f54c4165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 >> 172.21.15.96:0/3910260259 conn(0x7f54c406fbe0 msgr2=0x7f54c40b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 shutdown_connections 2024-09-21T02:10:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:13.975+0000 7f54c8c84640 1 -- 172.21.15.96:0/3910260259 wait complete. 2024-09-21T02:10:14.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:10:14.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 -- 172.21.15.96:0/2576942326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 msgr2=0x7f164c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 --2- 172.21.15.96:0/2576942326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f164c004c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f16580016f0 tx=0x7f165801c790 comp rx=0 tx=0).stop 2024-09-21T02:10:14.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 -- 172.21.15.96:0/2576942326 shutdown_connections 2024-09-21T02:10:14.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 --2- 172.21.15.96:0/2576942326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f164c004c20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 -- 172.21.15.96:0/2576942326 >> 172.21.15.96:0/2576942326 conn(0x7f166406fbe0 msgr2=0x7f166406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:14.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 -- 172.21.15.96:0/2576942326 shutdown_connections 2024-09-21T02:10:14.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.135+0000 7f1669872640 1 -- 172.21.15.96:0/2576942326 wait complete. 2024-09-21T02:10:14.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 Processor -- start 2024-09-21T02:10:14.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- start start 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f16641574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16640ca770 0x7f16640cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f166406ce90 0x7f166406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16640cef70 con 0x7f164c004810 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f16640cedf0 con 0x7f166406ce90 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f16640cf0f0 con 0x7f16640ca770 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f16641574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f16641574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57018/0 (socket says 172.21.15.96:57018) 2024-09-21T02:10:14.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 -- 172.21.15.96:0/2536540086 learned_addr learned my addr 172.21.15.96:0/2536540086 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f16637fe640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f166406ce90 0x7f166406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f16627fc640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16640ca770 0x7f16640cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 -- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16640ca770 msgr2=0x7f16640cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16640ca770 0x7f16640cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 -- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f166406ce90 msgr2=0x7f166406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f166406ce90 0x7f166406d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 -- 172.21.15.96:0/2536540086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16641f9470 con 0x7f164c004810 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f16637fe640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f166406ce90 0x7f166406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1662ffd640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f16641574e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1654011a80 tx=0x7f1654011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165400fe10 con 0x7f164c004810 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1654016070 con 0x7f164c004810 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16640cb170 con 0x7f164c004810 2024-09-21T02:10:14.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654012780 con 0x7f164c004810 2024-09-21T02:10:14.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166415df60 con 0x7f164c004810 2024-09-21T02:10:14.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.139+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16640d00e0 con 0x7f164c004810 2024-09-21T02:10:14.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.143+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1654018030 con 0x7f164c004810 2024-09-21T02:10:14.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.143+0000 7f164bfff640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1650073230 0x7f1650075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.143+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1654094f70 con 0x7f164c004810 2024-09-21T02:10:14.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.143+0000 7f16627fc640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1650073230 0x7f1650075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.143+0000 7f16627fc640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1650073230 0x7f1650075740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16580016c0 tx=0x7f1658019580 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:14.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.147+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f165405fb80 con 0x7f164c004810 2024-09-21T02:10:14.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.307+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1664072270 con 0x7f164c004810 2024-09-21T02:10:14.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f164bfff640 1 -- 172.21.15.96:0/2536540086 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18985 (secure 0 0 0) 0x7f165405f480 con 0x7f164c004810 2024-09-21T02:10:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:14.316 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":28,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:09:51.588360+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:09:08.825867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:09:09.856090+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T02:09:13.568880+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:10:14.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1650073230 msgr2=0x7f1650075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1650073230 0x7f1650075740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16580016c0 tx=0x7f1658019580 comp rx=0 tx=0).stop 2024-09-21T02:10:14.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 msgr2=0x7f16641574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f16641574e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1654011a80 tx=0x7f1654011f40 comp rx=0 tx=0).stop 2024-09-21T02:10:14.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 shutdown_connections 2024-09-21T02:10:14.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1650073230 0x7f1650075740 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f166406ce90 0x7f166406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16640ca770 0x7f16640cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 --2- 172.21.15.96:0/2536540086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c004810 0x7f16641574e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 >> 172.21.15.96:0/2536540086 conn(0x7f166406fbe0 msgr2=0x7f16640b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:14.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 shutdown_connections 2024-09-21T02:10:14.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.311+0000 7f1669872640 1 -- 172.21.15.96:0/2536540086 wait complete. 2024-09-21T02:10:14.337 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:10:14.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 -- 172.21.15.96:0/3562864454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac06e3e0 msgr2=0x7fdfac15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/3562864454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac06e3e0 0x7fdfac15bea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009840 tx=0x7fdfa001c820 comp rx=0 tx=0).stop 2024-09-21T02:10:14.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 -- 172.21.15.96:0/3562864454 shutdown_connections 2024-09-21T02:10:14.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/3562864454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac06e3e0 0x7fdfac15bea0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/3562864454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfac0ced00 0x7fdfac06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 -- 172.21.15.96:0/3562864454 >> 172.21.15.96:0/3562864454 conn(0x7fdfac06eea0 msgr2=0x7fdfac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:14.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 -- 172.21.15.96:0/3562864454 shutdown_connections 2024-09-21T02:10:14.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.479+0000 7fdfb23e8640 1 -- 172.21.15.96:0/3562864454 wait complete. 2024-09-21T02:10:14.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 Processor -- start 2024-09-21T02:10:14.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 -- start start 2024-09-21T02:10:14.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac06e3e0 0x7fdfac1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfac0ced00 0x7fdfac155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 0x7fdfac0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac168b60 con 0x7fdfac15c3e0 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1689e0 con 0x7fdfac06e3e0 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb23e8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac168ce0 con 0x7fdfac0ced00 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 0x7fdfac0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 0x7fdfac0b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57036/0 (socket says 172.21.15.96:57036) 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 -- 172.21.15.96:0/934338302 learned_addr learned my addr 172.21.15.96:0/934338302 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:14.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfabfff640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac06e3e0 0x7fdfac1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfab7fe640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfac0ced00 0x7fdfac155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 -- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfac0ced00 msgr2=0x7fdfac155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfac0ced00 0x7fdfac155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 -- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac06e3e0 msgr2=0x7fdfac1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac06e3e0 0x7fdfac1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 -- 172.21.15.96:0/934338302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac1593a0 con 0x7fdfac15c3e0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfabfff640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac06e3e0 0x7fdfac1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfb0985640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 0x7fdfac0b45b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400edd0 tx=0x7fdfa400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4016070 con 0x7fdfac15c3e0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa400ce10 con 0x7fdfac15c3e0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.483+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40116d0 con 0x7fdfac15c3e0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac159680 con 0x7fdfac15c3e0 2024-09-21T02:10:14.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac159c40 con 0x7fdfac15c3e0 2024-09-21T02:10:14.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac15bea0 con 0x7fdfac15c3e0 2024-09-21T02:10:14.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdfa4019070 con 0x7fdfac15c3e0 2024-09-21T02:10:14.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfa8ff9640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdf98073200 0x7fdf98075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdfa4096ac0 con 0x7fdfac15c3e0 2024-09-21T02:10:14.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfabfff640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdf98073200 0x7fdf98075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.487+0000 7fdfabfff640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdf98073200 0x7fdf98075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c005fd0 tx=0x7fdf9c005dc0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:14.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.491+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdfa4020030 con 0x7fdfac15c3e0 2024-09-21T02:10:14.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.691+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdfac15a440 con 0x7fdfac15c3e0 2024-09-21T02:10:14.698 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-21T02:10:14.691+0000 7f19643c0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:10:14.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.695+0000 7fdfa8ff9640 1 -- 172.21.15.96:0/934338302 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fdfac06e7f0 con 0x7fdfac15c3e0 2024-09-21T02:10:14.702 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdf98073200 msgr2=0x7fdf98075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdf98073200 0x7fdf98075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c005fd0 tx=0x7fdf9c005dc0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 msgr2=0x7fdfac0b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 0x7fdfac0b45b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400edd0 tx=0x7fdfa400eec0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 shutdown_connections 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdf98073200 0x7fdf98075710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac15c3e0 0x7fdfac0b45b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfac0ced00 0x7fdfac155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 --2- 172.21.15.96:0/934338302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac06e3e0 0x7fdfac1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 >> 172.21.15.96:0/934338302 conn(0x7fdfac06eea0 msgr2=0x7fdfac15d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:14.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 shutdown_connections 2024-09-21T02:10:14.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.699+0000 7fdfb23e8640 1 -- 172.21.15.96:0/934338302 wait complete. 2024-09-21T02:10:14.707 INFO:tasks.ceph.mds.c.smithi096.stderr: -4781> 2024-09-21T02:09:15.460+0000 7efd5ad08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:10:14.726 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:10:14.766 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2024-09-21T02:10:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2458965457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 msgr2=0x7f48e00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2458965457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e00cf010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009840 tx=0x7f48d001c790 comp rx=0 tx=0).stop 2024-09-21T02:10:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2458965457 shutdown_connections 2024-09-21T02:10:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2458965457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e00cf010 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2458965457 >> 172.21.15.96:0/2458965457 conn(0x7f48e006eea0 msgr2=0x7f48e006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2458965457 shutdown_connections 2024-09-21T02:10:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.875+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2458965457 wait complete. 2024-09-21T02:10:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 Processor -- start 2024-09-21T02:10:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 -- start start 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e006e2c0 0x7f48e00b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e01f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48e00cec00 0x7f48e01fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48e016ae80 con 0x7f48e00c53a0 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48e016ad00 con 0x7f48e006e2c0 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f48e016b000 con 0x7f48e00cec00 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e01f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e01f9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57074/0 (socket says 172.21.15.96:57074) 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 -- 172.21.15.96:0/2343513641 learned_addr learned my addr 172.21.15.96:0/2343513641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48de59c640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e006e2c0 0x7f48e00b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 -- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48e00cec00 msgr2=0x7f48e01fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48e00cec00 0x7f48e01fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 -- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e006e2c0 msgr2=0x7f48e00b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e006e2c0 0x7f48e00b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 -- 172.21.15.96:0/2343513641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e01fc680 con 0x7f48e00c53a0 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48de59c640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e006e2c0 0x7f48e00b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ddd9b640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e01f9610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c3c0 tx=0x7f48d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017070 con 0x7f48e00c53a0 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48d4003600 con 0x7f48e00c53a0 2024-09-21T02:10:14.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e01f8b50 con 0x7f48e00c53a0 2024-09-21T02:10:14.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4003a50 con 0x7f48e00c53a0 2024-09-21T02:10:14.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.879+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e01fce70 con 0x7f48e00c53a0 2024-09-21T02:10:14.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.883+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e00cf010 con 0x7f48e00c53a0 2024-09-21T02:10:14.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.883+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48d401a070 con 0x7f48e00c53a0 2024-09-21T02:10:14.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.883+0000 7f48ceffd640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f48c0073200 0x7f48c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:14.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.883+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7f48d4093850 con 0x7f48e00c53a0 2024-09-21T02:10:14.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.883+0000 7f48de59c640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f48c0073200 0x7f48c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:14.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.883+0000 7f48de59c640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f48c0073200 0x7f48c0075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48d8007fd0 tx=0x7f48d8009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:14.887+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f48d405e530 con 0x7f48e00c53a0 2024-09-21T02:10:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.095+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f48e0073310 con 0x7f48e00c53a0 2024-09-21T02:10:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.699+0000 7f48ceffd640 1 -- 172.21.15.96:0/2343513641 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f48d405de30 con 0x7f48e00c53a0 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f48c0073200 msgr2=0x7f48c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f48c0073200 0x7f48c0075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48d8007fd0 tx=0x7f48d8009040 comp rx=0 tx=0).stop 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 msgr2=0x7f48e01f9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e01f9610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c3c0 tx=0x7f48d400c880 comp rx=0 tx=0).stop 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 shutdown_connections 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f48c0073200 0x7f48c0075710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f48e00cec00 0x7f48e01fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e00c53a0 0x7f48e01f9610 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 --2- 172.21.15.96:0/2343513641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e006e2c0 0x7f48e00b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 >> 172.21.15.96:0/2343513641 conn(0x7f48e006eea0 msgr2=0x7f48e00b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 shutdown_connections 2024-09-21T02:10:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.703+0000 7f48e4a1f640 1 -- 172.21.15.96:0/2343513641 wait complete. 2024-09-21T02:10:15.736 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 -- 172.21.15.96:0/1960695375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59406f700 msgr2=0x7fa59406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 --2- 172.21.15.96:0/1960695375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59406f700 0x7fa59406fb10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009900 tx=0x7fa57c01c640 comp rx=0 tx=0).stop 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 -- 172.21.15.96:0/1960695375 shutdown_connections 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 --2- 172.21.15.96:0/1960695375 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa5940cd370 0x7fa5940cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 --2- 172.21.15.96:0/1960695375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59406f700 0x7fa59406fb10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 -- 172.21.15.96:0/1960695375 >> 172.21.15.96:0/1960695375 conn(0x7fa5940d05a0 msgr2=0x7fa5940d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:15.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 -- 172.21.15.96:0/1960695375 shutdown_connections 2024-09-21T02:10:15.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 -- 172.21.15.96:0/1960695375 wait complete. 2024-09-21T02:10:15.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 Processor -- start 2024-09-21T02:10:15.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.939+0000 7fa599bd4640 1 -- start start 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa59406f700 0x7fa5941f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5940cd370 0x7fa59415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 0x7fa594156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941659d0 con 0x7fa59415aa60 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa594165850 con 0x7fa5940cd370 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa594165b50 con 0x7fa59406f700 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 0x7fa594156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 0x7fa594156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57084/0 (socket says 172.21.15.96:57084) 2024-09-21T02:10:15.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 -- 172.21.15.96:0/2666160981 learned_addr learned my addr 172.21.15.96:0/2666160981 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa5937fe640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa59406f700 0x7fa5941f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa58affd640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5940cd370 0x7fa59415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 -- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa59406f700 msgr2=0x7fa5941f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa59406f700 0x7fa5941f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 -- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5940cd370 msgr2=0x7fa59415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5940cd370 0x7fa59415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 -- 172.21.15.96:0/2666160981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5941fa1b0 con 0x7fa59415aa60 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa5937fe640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa59406f700 0x7fa5941f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa58affd640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5940cd370 0x7fa59415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa593fff640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 0x7fa594156bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00ca60 tx=0x7fa58c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c007d90 con 0x7fa59415aa60 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa58c004c20 con 0x7fa59415aa60 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59413ff50 con 0x7fa59415aa60 2024-09-21T02:10:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c005280 con 0x7fa59415aa60 2024-09-21T02:10:15.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa594140510 con 0x7fa59415aa60 2024-09-21T02:10:15.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59406fb10 con 0x7fa59415aa60 2024-09-21T02:10:15.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa58c005420 con 0x7fa59415aa60 2024-09-21T02:10:15.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa590ff9640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa574073200 0x7fa574075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:15.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7fa58c094d40 con 0x7fa59415aa60 2024-09-21T02:10:15.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.943+0000 7fa5937fe640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa574073200 0x7fa574075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:15.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.947+0000 7fa5937fe640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa574073200 0x7fa574075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa58400a850 tx=0x7fa584008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:15.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:15.947+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa58c014030 con 0x7fa59415aa60 2024-09-21T02:10:16.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.111+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa59413fce0 con 0x7fa59415aa60 2024-09-21T02:10:16.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.711+0000 7fa590ff9640 1 -- 172.21.15.96:0/2666160981 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7fa58c05f990 con 0x7fa59415aa60 2024-09-21T02:10:16.715 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:10:16.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.711+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa574073200 msgr2=0x7fa574075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:16.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.711+0000 7fa599bd4640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa574073200 0x7fa574075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa58400a850 tx=0x7fa584008040 comp rx=0 tx=0).stop 2024-09-21T02:10:16.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.711+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 msgr2=0x7fa594156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:16.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.711+0000 7fa599bd4640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 0x7fa594156bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00ca60 tx=0x7fa58c00cf20 comp rx=0 tx=0).stop 2024-09-21T02:10:16.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 shutdown_connections 2024-09-21T02:10:16.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa574073200 0x7fa574075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:16.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa59415aa60 0x7fa594156bb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:16.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5940cd370 0x7fa59415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:16.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 --2- 172.21.15.96:0/2666160981 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa59406f700 0x7fa5941f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:16.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 >> 172.21.15.96:0/2666160981 conn(0x7fa5940d05a0 msgr2=0x7fa594153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:16.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 shutdown_connections 2024-09-21T02:10:16.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.715+0000 7fa599bd4640 1 -- 172.21.15.96:0/2666160981 wait complete. 2024-09-21T02:10:16.747 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:10:16.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3639e640 1 -- 172.21.15.96:0/1635800718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f18003a40 msgr2=0x7f8f18003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:16.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3639e640 1 --2- 172.21.15.96:0/1635800718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f18003a40 0x7f8f18003eb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8f200016f0 tx=0x7f8f2001c770 comp rx=0 tx=0).stop 2024-09-21T02:10:16.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3493b640 1 -- 172.21.15.96:0/1635800718 reap_dead start 2024-09-21T02:10:16.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3639e640 1 -- 172.21.15.96:0/1635800718 shutdown_connections 2024-09-21T02:10:16.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3639e640 1 -- 172.21.15.96:0/1635800718 >> 172.21.15.96:0/1635800718 conn(0x7f8f300d2060 msgr2=0x7f8f300d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:16.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3639e640 1 -- 172.21.15.96:0/1635800718 shutdown_connections 2024-09-21T02:10:16.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.907+0000 7f8f3639e640 1 -- 172.21.15.96:0/1635800718 wait complete. 2024-09-21T02:10:16.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 Processor -- start 2024-09-21T02:10:16.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 -- start start 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f18003a40 0x7f8f30140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f3006f730 0x7f8f300ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 0x7f8f300cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30168c40 con 0x7f8f3006fb40 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30168ac0 con 0x7f8f18003a40 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8f30168dc0 con 0x7f8f3006f730 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 0x7f8f300cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:16.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 0x7f8f300cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57456/0 (socket says 172.21.15.96:57456) 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 -- 172.21.15.96:0/2131983504 learned_addr learned my addr 172.21.15.96:0/2131983504 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f2f7fe640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f3006f730 0x7f8f300ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f2ffff640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f18003a40 0x7f8f30140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 -- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f3006f730 msgr2=0x7f8f300ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f3006f730 0x7f8f300ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 -- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f18003a40 msgr2=0x7f8f30140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f18003a40 0x7f8f30140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 -- 172.21.15.96:0/2131983504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f301410a0 con 0x7f8f3006fb40 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f2ffff640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f18003a40 0x7f8f30140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3493b640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 0x7f8f300cd110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f30145af0 tx=0x7f8f2800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:16.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28018070 con 0x7f8f3006fb40 2024-09-21T02:10:16.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f28003c90 con 0x7f8f3006fb40 2024-09-21T02:10:16.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f301fc3a0 con 0x7f8f3006fb40 2024-09-21T02:10:16.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28004290 con 0x7f8f3006fb40 2024-09-21T02:10:16.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.911+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f301fc960 con 0x7f8f3006fb40 2024-09-21T02:10:16.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.915+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3006dfa0 con 0x7f8f3006fb40 2024-09-21T02:10:16.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.915+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8f2801b070 con 0x7f8f3006fb40 2024-09-21T02:10:16.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.915+0000 7f8f2cff9640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8f1c073230 0x7f8f1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:16.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.915+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6383+0+0 (secure 0 0 0) 0x7f8f28093a70 con 0x7f8f3006fb40 2024-09-21T02:10:16.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.915+0000 7f8f2ffff640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8f1c073230 0x7f8f1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:16.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.915+0000 7f8f2ffff640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8f1c073230 0x7f8f1c075740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f200016c0 tx=0x7f8f200a8040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:16.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:16.919+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8f2805e7f0 con 0x7f8f3006fb40 2024-09-21T02:10:17.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.091+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8f300b6890 con 0x7f8f3006fb40 2024-09-21T02:10:17.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.719+0000 7f8f2cff9640 1 -- 172.21.15.96:0/2131983504 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f8f2805e0f0 con 0x7f8f3006fb40 2024-09-21T02:10:17.725 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-21T02:10:17.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8f1c073230 msgr2=0x7f8f1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:17.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8f1c073230 0x7f8f1c075740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f200016c0 tx=0x7f8f200a8040 comp rx=0 tx=0).stop 2024-09-21T02:10:17.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 msgr2=0x7f8f300cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:17.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 0x7f8f300cd110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f30145af0 tx=0x7f8f2800cb80 comp rx=0 tx=0).stop 2024-09-21T02:10:17.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 shutdown_connections 2024-09-21T02:10:17.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8f1c073230 0x7f8f1c075740 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f3006fb40 0x7f8f300cd110 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f3006f730 0x7f8f300ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 --2- 172.21.15.96:0/2131983504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f18003a40 0x7f8f30140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 >> 172.21.15.96:0/2131983504 conn(0x7f8f300d2060 msgr2=0x7f8f30160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:17.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 shutdown_connections 2024-09-21T02:10:17.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.723+0000 7f8f3639e640 1 -- 172.21.15.96:0/2131983504 wait complete. 2024-09-21T02:10:17.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T02:10:17.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f6861b6c640 1 -- 172.21.15.96:0/2440535808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 msgr2=0x7f6844003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:17.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2440535808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f6844003eb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f684c00d370 tx=0x7f684c01e2b0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f685bfff640 1 -- 172.21.15.96:0/2440535808 reap_dead start 2024-09-21T02:10:17.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f6861b6c640 1 -- 172.21.15.96:0/2440535808 shutdown_connections 2024-09-21T02:10:17.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f6861b6c640 1 -- 172.21.15.96:0/2440535808 >> 172.21.15.96:0/2440535808 conn(0x7f685c0d05a0 msgr2=0x7f685c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:17.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f6861b6c640 1 -- 172.21.15.96:0/2440535808 shutdown_connections 2024-09-21T02:10:17.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.911+0000 7f6861b6c640 1 -- 172.21.15.96:0/2440535808 wait complete. 2024-09-21T02:10:17.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 Processor -- start 2024-09-21T02:10:17.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- start start 2024-09-21T02:10:17.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f685c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:17.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f685c163c10 0x7f685c164020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:17.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685c1fbe90 0x7f685c1fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f685c0cdca0 con 0x7f6844003a40 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f685c0cdb20 con 0x7f685c163c10 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f685c0cde20 con 0x7f685c1fbe90 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f685c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f685c151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57484/0 (socket says 172.21.15.96:57484) 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 -- 172.21.15.96:0/2048444379 learned_addr learned my addr 172.21.15.96:0/2048444379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685bfff640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685c1fbe90 0x7f685c1fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685affd640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f685c163c10 0x7f685c164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 -- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685c1fbe90 msgr2=0x7f685c1fc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685c1fbe90 0x7f685c1fc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 -- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f685c163c10 msgr2=0x7f685c164020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f685c163c10 0x7f685c164020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 -- 172.21.15.96:0/2048444379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c164560 con 0x7f6844003a40 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685bfff640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685c1fbe90 0x7f685c1fc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f685b7fe640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f685c151e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f684c01c9b0 tx=0x7f684c006cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c006090 con 0x7f6844003a40 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f684c006420 con 0x7f6844003a40 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c1fcbc0 con 0x7f6844003a40 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c003860 con 0x7f6844003a40 2024-09-21T02:10:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c1f8e40 con 0x7f6844003a40 2024-09-21T02:10:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.915+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c0b4420 con 0x7f6844003a40 2024-09-21T02:10:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.919+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f684c003a00 con 0x7f6844003a40 2024-09-21T02:10:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.919+0000 7f6843fff640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6848073200 0x7f6848075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.919+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7f684c12e850 con 0x7f6844003a40 2024-09-21T02:10:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.919+0000 7f685affd640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6848073200 0x7f6848075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.919+0000 7f685affd640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6848073200 0x7f6848075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c10 tx=0x7f6850006a50 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:17.923+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f685c164560 con 0x7f6844003a40 2024-09-21T02:10:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.095+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f685c159e00 con 0x7f6844003a40 2024-09-21T02:10:18.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.727+0000 7f6843fff640 1 -- 172.21.15.96:0/2048444379 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f684c018060 con 0x7f6844003a40 2024-09-21T02:10:18.734 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data_ec' removed 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6848073200 msgr2=0x7f6848075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6848073200 0x7f6848075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c10 tx=0x7f6850006a50 comp rx=0 tx=0).stop 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 msgr2=0x7f685c151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f685c151e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f684c01c9b0 tx=0x7f684c006cb0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 shutdown_connections 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6848073200 0x7f6848075710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685c1fbe90 0x7f685c1fc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f685c163c10 0x7f685c164020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 --2- 172.21.15.96:0/2048444379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6844003a40 0x7f685c151e30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 >> 172.21.15.96:0/2048444379 conn(0x7f685c0d05a0 msgr2=0x7f685c155690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:18.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 shutdown_connections 2024-09-21T02:10:18.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.735+0000 7f6861b6c640 1 -- 172.21.15.96:0/2048444379 wait complete. 2024-09-21T02:10:18.768 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:10:18.768 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:10:18.769 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:10:18.769 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:18.769 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:10:18.769 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:10:18.769 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:10:18.769 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:18.769 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:10:18.769 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:10:18.770 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:10:18.770 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:18.770 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:10:18.770 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:10:18.770 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:10:18.770 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:10:18.782 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:10:18.782 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:10:18.785 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:10:18.785 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:10:18.791 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:10:18.791 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:10:18.792 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:10:18.792 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:10:18.827 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:10:18.827 DEBUG:teuthology.parallel:result is None 2024-09-21T02:10:18.828 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:10:18.828 DEBUG:teuthology.parallel:result is None 2024-09-21T02:10:18.836 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:10:18.836 DEBUG:teuthology.parallel:result is None 2024-09-21T02:10:18.837 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:10:18.837 DEBUG:teuthology.parallel:result is None 2024-09-21T02:10:18.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:10:18.920 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2024-09-21T02:10:18.921 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-21T02:10:18.936 INFO:tasks.ceph.mds.d.smithi122.stdout:starting mds.d at 2024-09-21T02:10:18.937 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-09-21T02:10:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 -- 172.21.15.96:0/1154428312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 msgr2=0x7f61c4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/1154428312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61c4004c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006fd0 tx=0x7f61c001f6a0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 -- 172.21.15.96:0/1154428312 shutdown_connections 2024-09-21T02:10:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/1154428312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61c4004c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 -- 172.21.15.96:0/1154428312 >> 172.21.15.96:0/1154428312 conn(0x7f61d80d1b10 msgr2=0x7f61d80d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 -- 172.21.15.96:0/1154428312 shutdown_connections 2024-09-21T02:10:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 -- 172.21.15.96:0/1154428312 wait complete. 2024-09-21T02:10:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 Processor -- start 2024-09-21T02:10:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 -- start start 2024-09-21T02:10:18.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61d81f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:18.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61d80ce1b0 0x7f61d8152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:18.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.991+0000 7f61dd7f3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61d815aa50 0x7f61d815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61dd7f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61d815fbe0 con 0x7f61c4004810 2024-09-21T02:10:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61dd7f3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f61d80cfc30 con 0x7f61d815aa50 2024-09-21T02:10:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61dd7f3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f61d815fd60 con 0x7f61d80ce1b0 2024-09-21T02:10:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61d81f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61d81f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57548/0 (socket says 172.21.15.96:57548) 2024-09-21T02:10:19.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 -- 172.21.15.96:0/656589932 learned_addr learned my addr 172.21.15.96:0/656589932 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:19.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d77fe640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61d815aa50 0x7f61d815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61ce7fc640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61d80ce1b0 0x7f61d8152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 -- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61d80ce1b0 msgr2=0x7f61d8152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61d80ce1b0 0x7f61d8152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 -- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61d815aa50 msgr2=0x7f61d815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61d815aa50 0x7f61d815ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 -- 172.21.15.96:0/656589932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d8153020 con 0x7f61c4004810 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61d6ffd640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61d81f6970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61c8011a10 tx=0x7f61c8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c801e070 con 0x7f61c4004810 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61c800fbb0 con 0x7f61c4004810 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8019410 con 0x7f61c4004810 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d81599b0 con 0x7f61c4004810 2024-09-21T02:10:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d8159f70 con 0x7f61c4004810 2024-09-21T02:10:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d815f450 con 0x7f61c4004810 2024-09-21T02:10:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f61c8021070 con 0x7f61c4004810 2024-09-21T02:10:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61cffff640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f61b8073200 0x7f61b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f61c8095010 con 0x7f61c4004810 2024-09-21T02:10:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61ce7fc640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f61b8073200 0x7f61b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.995+0000 7f61ce7fc640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f61b8073200 0x7f61b8075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f61c00b0000 tx=0x7f61c0005040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:18.999+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f61c805fc30 con 0x7f61c4004810 2024-09-21T02:10:19.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.183+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f61d8157910 con 0x7f61c4004810 2024-09-21T02:10:19.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.751+0000 7f61cffff640 1 -- 172.21.15.96:0/656589932 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f61c8064660 con 0x7f61c4004810 2024-09-21T02:10:19.759 INFO:teuthology.orchestra.run.smithi096.stderr: removed all blocklist entries 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f61b8073200 msgr2=0x7f61b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f61b8073200 0x7f61b8075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f61c00b0000 tx=0x7f61c0005040 comp rx=0 tx=0).stop 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 msgr2=0x7f61d81f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61d81f6970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61c8011a10 tx=0x7f61c8011ed0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 shutdown_connections 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f61b8073200 0x7f61b8075710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61d815aa50 0x7f61d815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61d80ce1b0 0x7f61d8152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 --2- 172.21.15.96:0/656589932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c4004810 0x7f61d81f6970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 >> 172.21.15.96:0/656589932 conn(0x7f61d80d1b10 msgr2=0x7f61d8161480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 shutdown_connections 2024-09-21T02:10:19.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.759+0000 7f61dd7f3640 1 -- 172.21.15.96:0/656589932 wait complete. 2024-09-21T02:10:19.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 -- 172.21.15.96:0/1183151616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 msgr2=0x7f601006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1183151616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601006dea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f60100b61b0 tx=0x7f600001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 -- 172.21.15.96:0/1183151616 shutdown_connections 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1183151616 >> v2:172.21.15.122:3300/0 conn(0x7f601006e3e0 0x7f601015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1183151616 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f601015c3e0 0x7f60100cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1183151616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601006dea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 -- 172.21.15.96:0/1183151616 >> 172.21.15.96:0/1183151616 conn(0x7f601006eea0 msgr2=0x7f601006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 -- 172.21.15.96:0/1183151616 shutdown_connections 2024-09-21T02:10:19.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.951+0000 7f6015e9f640 1 -- 172.21.15.96:0/1183151616 wait complete. 2024-09-21T02:10:19.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 Processor -- start 2024-09-21T02:10:19.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- start start 2024-09-21T02:10:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f601006e3e0 0x7f6010167b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f601015c3e0 0x7f601015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60100cab40 con 0x7f60100ced00 2024-09-21T02:10:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60100ca9c0 con 0x7f601015c3e0 2024-09-21T02:10:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f60100cacc0 con 0x7f601006e3e0 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57566/0 (socket says 172.21.15.96:57566) 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 -- 172.21.15.96:0/1029981045 learned_addr learned my addr 172.21.15.96:0/1029981045 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f600f7fe640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f601006e3e0 0x7f6010167b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f600ffff640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f601015c3e0 0x7f601015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 -- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f601006e3e0 msgr2=0x7f6010167b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f601006e3e0 0x7f6010167b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 -- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f601015c3e0 msgr2=0x7f601015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f601015c3e0 0x7f601015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 -- 172.21.15.96:0/1029981045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6010160d00 con 0x7f60100ced00 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6007fff640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601015a4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800d700 tx=0x7f5ff800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8017070 con 0x7f60100ced00 2024-09-21T02:10:19.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ff8009b10 con 0x7f60100ced00 2024-09-21T02:10:19.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6010160fe0 con 0x7f60100ced00 2024-09-21T02:10:19.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8010460 con 0x7f60100ced00 2024-09-21T02:10:19.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101615a0 con 0x7f60100ced00 2024-09-21T02:10:19.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.955+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60100cb260 con 0x7f60100ced00 2024-09-21T02:10:19.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.959+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ff801a070 con 0x7f60100ced00 2024-09-21T02:10:19.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.959+0000 7f600cff9640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ff0073200 0x7f5ff0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.959+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5ff8094510 con 0x7f60100ced00 2024-09-21T02:10:19.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.959+0000 7f600f7fe640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ff0073200 0x7f5ff0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.959+0000 7f600f7fe640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ff0073200 0x7f5ff0075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60000af000 tx=0x7f600001b100 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:19.963+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5ff805f860 con 0x7f60100ced00 2024-09-21T02:10:20.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.191+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f60100b09f0 con 0x7f60100ced00 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.191+0000 7f600cff9640 1 -- 172.21.15.96:0/1029981045 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+6645 (secure 0 0 0) 0x7f5ff805f160 con 0x7f60100ced00 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_dump": [ 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.a", 2024-09-21T02:10:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mkCWlDRAAzdu0wONk/0Lupo5cZ3NHzQ==", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.b", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mnXSzGhAA0dpvCpA6xxnAsiJJ20Y1Qg==", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.c", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mO6U8FRAAXyQxOIliLEhF5gm7p8ZHmw==", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.d", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mMpzaIhAAY0BZMv3gutB1Aqe1bPOGgQ==", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.0", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCHKu5mu92zORAAaLLvKlmkKyWEyhVMFOUsgQ==", 2024-09-21T02:10:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.1", 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCLKu5mqWPTDBAA6vUHoA/PGj0RhabCpEzrHQ==", 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.2", 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCOKu5mkhNlGxAA3iSsM1nh92w+bNjtOGrK5g==", 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.3", 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCRKu5mUT5xKhAAu1+5N4qMqW7sUyyk5SmOPw==", 2024-09-21T02:10:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.4", 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCXKu5mVAaYLRAA07ptXx3hhzgxcqbcFPe5XQ==", 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.5", 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCbKu5mBfaZABAA0RK0Pv4EC8imtRDSkCevKg==", 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.6", 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCeKu5m1+02EhAAiWYPYmpourCdhLuJHMNoRQ==", 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.7", 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQChKu5mfXOxIBAAz1mbwiOSpnL+a+zCa4krBA==", 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.0", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5my+CoJxAAog4sBSDU0v9/F60t9/U1nQ==", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.1", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mojXOMBAAy74zTgdiJ5bDnXgc9VvuNg==", 2024-09-21T02:10:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-21T02:10:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-21T02:10:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-21T02:10:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.2", 2024-09-21T02:10:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mZqfSKxAAaIcNdXRvmymRHPV2adIDyg==", 2024-09-21T02:10:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.3", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5m8Ud8NRAAhHIgwf6D9G3sWEOKFyKVmA==", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-21T02:10:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.admin", 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCDKu5mskKQLBAAY/5fVutYOm2AszxhJIWVDA==", 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-mds", 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCsKu5m13MSNRAAe9BAGXZNXDRz4M3AI6vOdw==", 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCsKu5mFpcSNRAAd07B5+iuNa7+Wj5crjEuUA==", 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-osd", 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCsKu5m6K8SNRAAk5JRWUQBRubJ8Kb8Snu8EQ==", 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCsKu5mrcsSNRAAs6WrVSgsPLXoETEaj/qDKg==", 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCsKu5m4uMSNRAAkxIwntGIWPTc+QX3TwmdiQ==", 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCsKu5m9fwSNRAAoGG2rivHVIaNwVvZd2T2GQ==", 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mgr.x", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mLC2wCBAAZw8at6A2SvR34GvkK2hEUQ==", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile mgr", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mgr.y", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQCEKu5mC8rIAxAAcgR6MpZGGj2MxGPuYVtgbw==", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-21T02:10:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile mgr", 2024-09-21T02:10:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-21T02:10:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-21T02:10:20.216 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ff0073200 msgr2=0x7f5ff0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ff0073200 0x7f5ff0075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60000af000 tx=0x7f600001b100 comp rx=0 tx=0).stop 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 msgr2=0x7f601015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601015a4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800d700 tx=0x7f5ff800dbc0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 shutdown_connections 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f5ff0073200 0x7f5ff0075710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f601015c3e0 0x7f601015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60100ced00 0x7f601015a4b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 --2- 172.21.15.96:0/1029981045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f601006e3e0 0x7f6010167b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 >> 172.21.15.96:0/1029981045 conn(0x7f601006eea0 msgr2=0x7f60100afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:20.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 shutdown_connections 2024-09-21T02:10:20.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.195+0000 7f6015e9f640 1 -- 172.21.15.96:0/1029981045 wait complete. 2024-09-21T02:10:20.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T02:10:20.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f47606b2640 1 -- 172.21.15.96:0/3460691378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 msgr2=0x7f4744004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f47606b2640 1 --2- 172.21.15.96:0/3460691378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f4744004c20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4748009b60 tx=0x7f474801d2b0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f475ec4f640 1 -- 172.21.15.96:0/3460691378 reap_dead start 2024-09-21T02:10:20.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f47606b2640 1 -- 172.21.15.96:0/3460691378 shutdown_connections 2024-09-21T02:10:20.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f47606b2640 1 -- 172.21.15.96:0/3460691378 >> 172.21.15.96:0/3460691378 conn(0x7f47580d1130 msgr2=0x7f47580d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:20.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f47606b2640 1 -- 172.21.15.96:0/3460691378 shutdown_connections 2024-09-21T02:10:20.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.367+0000 7f47606b2640 1 -- 172.21.15.96:0/3460691378 wait complete. 2024-09-21T02:10:20.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 Processor -- start 2024-09-21T02:10:20.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 -- start start 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f47581e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47580ca930 0x7f4758146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f475814fbe0 0x7f47581e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4758160500 con 0x7f4744004810 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47580ce220 con 0x7f475814fbe0 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f47606b2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4758160680 con 0x7f47580ca930 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f47581e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f47581e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57584/0 (socket says 172.21.15.96:57584) 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 -- 172.21.15.96:0/767046144 learned_addr learned my addr 172.21.15.96:0/767046144 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:20.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475ec4f640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f475814fbe0 0x7f47581e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475dc4d640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47580ca930 0x7f4758146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 -- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47580ca930 msgr2=0x7f4758146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47580ca930 0x7f4758146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 -- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f475814fbe0 msgr2=0x7f47581e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f475814fbe0 0x7f47581e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 -- 172.21.15.96:0/767046144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580b4280 con 0x7f4744004810 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f475e44e640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f47581e82a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4750001780 tx=0x7f475000fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.371+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475001a070 con 0x7f4744004810 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4750006d90 con 0x7f4744004810 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47580b4560 con 0x7f4744004810 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000d8a0 con 0x7f4744004810 2024-09-21T02:10:20.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47580b4b20 con 0x7f4744004810 2024-09-21T02:10:20.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4758071210 con 0x7f4744004810 2024-09-21T02:10:20.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f475001d070 con 0x7f4744004810 2024-09-21T02:10:20.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f474effd640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f473c073200 0x7f473c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f47500967a0 con 0x7f4744004810 2024-09-21T02:10:20.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.375+0000 7f475dc4d640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f473c073200 0x7f473c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.379+0000 7f475dc4d640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f473c073200 0x7f473c075710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f47480af000 tx=0x7f474801a050 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:20.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.379+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4750024020 con 0x7f4744004810 2024-09-21T02:10:20.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.615+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f47580b1c60 con 0x7f4744004810 2024-09-21T02:10:20.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.619+0000 7f474effd640 1 -- 172.21.15.96:0/767046144 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v23) ==== 90+0+0 (secure 0 0 0) 0x7f47580cff70 con 0x7f4744004810 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f473c073200 msgr2=0x7f473c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f473c073200 0x7f473c075710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f47480af000 tx=0x7f474801a050 comp rx=0 tx=0).stop 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 msgr2=0x7f47581e82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f47581e82a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4750001780 tx=0x7f475000fb90 comp rx=0 tx=0).stop 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 shutdown_connections 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f473c073200 0x7f473c075710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f475814fbe0 0x7f47581e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47580ca930 0x7f4758146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 --2- 172.21.15.96:0/767046144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744004810 0x7f47581e82a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 >> 172.21.15.96:0/767046144 conn(0x7f47580d1130 msgr2=0x7f47580b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:20.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 shutdown_connections 2024-09-21T02:10:20.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.623+0000 7f47606b2640 1 -- 172.21.15.96:0/767046144 wait complete. 2024-09-21T02:10:20.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 -- 172.21.15.96:0/1649699642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec80d3470 msgr2=0x7f8ec8162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/1649699642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec80d3470 0x7f8ec8162b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009bc0 tx=0x7f8ec001cba0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 -- 172.21.15.96:0/1649699642 shutdown_connections 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/1649699642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec80d3470 0x7f8ec8162b20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/1649699642 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8ec80cf710 0x7f8ec815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 -- 172.21.15.96:0/1649699642 >> 172.21.15.96:0/1649699642 conn(0x7f8ec80d1b10 msgr2=0x7f8ec80d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 -- 172.21.15.96:0/1649699642 shutdown_connections 2024-09-21T02:10:20.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 -- 172.21.15.96:0/1649699642 wait complete. 2024-09-21T02:10:20.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 Processor -- start 2024-09-21T02:10:20.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.791+0000 7f8ece2c6640 1 -- start start 2024-09-21T02:10:20.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ec80cf710 0x7f8ec8155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8ec80d3470 0x7f8ec80cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 0x7f8ec80ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec816bdb0 con 0x7f8ec815f7d0 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec815fdf0 con 0x7f8ec80cf710 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec816bf30 con 0x7f8ec80d3470 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 0x7f8ec80ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 0x7f8ec80ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57590/0 (socket says 172.21.15.96:57590) 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 -- 172.21.15.96:0/4214195603 learned_addr learned my addr 172.21.15.96:0/4214195603 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ec7fff640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ec80cf710 0x7f8ec8155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ebf7fe640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8ec80d3470 0x7f8ec80cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 -- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8ec80d3470 msgr2=0x7f8ec80cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8ec80d3470 0x7f8ec80cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 -- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ec80cf710 msgr2=0x7f8ec8155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ec80cf710 0x7f8ec8155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 -- 172.21.15.96:0/4214195603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec81517e0 con 0x7f8ec815f7d0 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ecc863640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 0x7f8ec80ccfd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009b10 tx=0x7f8ec0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00bb070 con 0x7f8ec815f7d0 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ec0004e00 con 0x7f8ec815f7d0 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00b2040 con 0x7f8ec815f7d0 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec8159730 con 0x7f8ec815f7d0 2024-09-21T02:10:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec80b4d40 con 0x7f8ec815f7d0 2024-09-21T02:10:20.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec815f290 con 0x7f8ec815f7d0 2024-09-21T02:10:20.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.795+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ec00be070 con 0x7f8ec815f7d0 2024-09-21T02:10:20.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.799+0000 7f8ec57fa640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ea8073200 0x7f8ea8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.799+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f8ec012ddf0 con 0x7f8ec815f7d0 2024-09-21T02:10:20.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.799+0000 7f8ec7fff640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ea8073200 0x7f8ea8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.799+0000 7f8ec7fff640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ea8073200 0x7f8ea8075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80b7610 tx=0x7f8eb800b2e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:20.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:20.799+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8ec00a5e20 con 0x7f8ec815f7d0 2024-09-21T02:10:21.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.035+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8ec8151570 con 0x7f8ec815f7d0 2024-09-21T02:10:21.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.039+0000 7f8ec57fa640 1 -- 172.21.15.96:0/4214195603 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v24) ==== 90+0+0 (secure 0 0 0) 0x7f8ec0016790 con 0x7f8ec815f7d0 2024-09-21T02:10:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ea8073200 msgr2=0x7f8ea8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ea8073200 0x7f8ea8075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80b7610 tx=0x7f8eb800b2e0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 msgr2=0x7f8ec80ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 0x7f8ec80ccfd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009b10 tx=0x7f8ec0003f60 comp rx=0 tx=0).stop 2024-09-21T02:10:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 shutdown_connections 2024-09-21T02:10:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ea8073200 0x7f8ea8075710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec815f7d0 0x7f8ec80ccfd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8ec80d3470 0x7f8ec80cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 --2- 172.21.15.96:0/4214195603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ec80cf710 0x7f8ec8155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 >> 172.21.15.96:0/4214195603 conn(0x7f8ec80d1b10 msgr2=0x7f8ec8158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 shutdown_connections 2024-09-21T02:10:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.043+0000 7f8ece2c6640 1 -- 172.21.15.96:0/4214195603 wait complete. 2024-09-21T02:10:21.067 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T02:10:21.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 -- 172.21.15.96:0/876577846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff28406e2c0 msgr2=0x7ff2840c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 --2- 172.21.15.96:0/876577846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff28406e2c0 0x7ff2840c4580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff274009900 tx=0x7ff27401c870 comp rx=0 tx=0).stop 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 -- 172.21.15.96:0/876577846 shutdown_connections 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 --2- 172.21.15.96:0/876577846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff2840c4ac0 0x7ff2840c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 --2- 172.21.15.96:0/876577846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff28406e2c0 0x7ff2840c4580 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 -- 172.21.15.96:0/876577846 >> 172.21.15.96:0/876577846 conn(0x7ff28406eea0 msgr2=0x7ff28406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 -- 172.21.15.96:0/876577846 shutdown_connections 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.203+0000 7ff28aff9640 1 -- 172.21.15.96:0/876577846 wait complete. 2024-09-21T02:10:21.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 Processor -- start 2024-09-21T02:10:21.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 -- start start 2024-09-21T02:10:21.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff28406e2c0 0x7ff284154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 0x7ff284150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2840cdbf0 0x7ff28415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2840cf0a0 con 0x7ff2840c4ac0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2840cef20 con 0x7ff2840cdbf0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff2840cf220 con 0x7ff28406e2c0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 0x7ff284150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 0x7ff284150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57608/0 (socket says 172.21.15.96:57608) 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 -- 172.21.15.96:0/3149544544 learned_addr learned my addr 172.21.15.96:0/3149544544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff288d95640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff28406e2c0 0x7ff284154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff289596640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2840cdbf0 0x7ff28415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 -- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff28406e2c0 msgr2=0x7ff284154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff28406e2c0 0x7ff284154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 -- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2840cdbf0 msgr2=0x7ff28415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2840cdbf0 0x7ff28415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 -- 172.21.15.96:0/3149544544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff284151400 con 0x7ff2840c4ac0 2024-09-21T02:10:21.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff289596640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2840cdbf0 0x7ff28415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff283fff640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 0x7ff284150ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d680 tx=0x7ff27800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x7ff2840c4ac0 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff278005310 con 0x7ff2840c4ac0 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2840c8870 con 0x7ff2840c4ac0 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780097c0 con 0x7ff2840c4ac0 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.207+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2840c71e0 con 0x7ff2840c4ac0 2024-09-21T02:10:21.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.211+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2840c4580 con 0x7ff2840c4ac0 2024-09-21T02:10:21.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.211+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff278009960 con 0x7ff2840c4ac0 2024-09-21T02:10:21.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.211+0000 7ff2817fa640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff270073200 0x7ff270075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.211+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff2780943b0 con 0x7ff2840c4ac0 2024-09-21T02:10:21.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.211+0000 7ff288d95640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff270073200 0x7ff270075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.211+0000 7ff288d95640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff270073200 0x7ff270075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff2740af000 tx=0x7ff27401b0b0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:21.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.215+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff27805f700 con 0x7ff2840c4ac0 2024-09-21T02:10:21.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.431+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7ff2840c8600 con 0x7ff2840c4ac0 2024-09-21T02:10:21.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.435+0000 7ff2817fa640 1 -- 172.21.15.96:0/3149544544 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v25) ==== 90+0+0 (secure 0 0 0) 0x7ff27805f000 con 0x7ff2840c4ac0 2024-09-21T02:10:21.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff270073200 msgr2=0x7ff270075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff270073200 0x7ff270075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff2740af000 tx=0x7ff27401b0b0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 msgr2=0x7ff284150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 0x7ff284150ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d680 tx=0x7ff27800db40 comp rx=0 tx=0).stop 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 shutdown_connections 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff270073200 0x7ff270075710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2840cdbf0 0x7ff28415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2840c4ac0 0x7ff284150ec0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 --2- 172.21.15.96:0/3149544544 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff28406e2c0 0x7ff284154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 >> 172.21.15.96:0/3149544544 conn(0x7ff28406eea0 msgr2=0x7ff284153560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:21.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 shutdown_connections 2024-09-21T02:10:21.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.439+0000 7ff28aff9640 1 -- 172.21.15.96:0/3149544544 wait complete. 2024-09-21T02:10:21.462 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 -- 172.21.15.96:0/3726170642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600cf710 msgr2=0x7fb86015f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 --2- 172.21.15.96:0/3726170642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600cf710 0x7fb86015f290 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb850009900 tx=0x7fb85001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 -- 172.21.15.96:0/3726170642 shutdown_connections 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 --2- 172.21.15.96:0/3726170642 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8600d3470 0x7fb860162b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 --2- 172.21.15.96:0/3726170642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600cf710 0x7fb86015f290 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 -- 172.21.15.96:0/3726170642 >> 172.21.15.96:0/3726170642 conn(0x7fb8600d1b10 msgr2=0x7fb8600d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:21.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 -- 172.21.15.96:0/3726170642 shutdown_connections 2024-09-21T02:10:21.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.603+0000 7fb864d47640 1 -- 172.21.15.96:0/3726170642 wait complete. 2024-09-21T02:10:21.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 Processor -- start 2024-09-21T02:10:21.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- start start 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8600cf710 0x7fb8600cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 0x7fb8600b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86015f7d0 0x7fb8600b9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb86016bdb0 con 0x7fb8600d3470 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb86015fdf0 con 0x7fb86015f7d0 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb86016bf30 con 0x7fb8600cf710 2024-09-21T02:10:21.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 0x7fb8600b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 0x7fb8600b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57622/0 (socket says 172.21.15.96:57622) 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 -- 172.21.15.96:0/360785802 learned_addr learned my addr 172.21.15.96:0/360785802 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb85e59c640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8600cf710 0x7fb8600cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb85ed9d640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86015f7d0 0x7fb8600b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 -- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8600cf710 msgr2=0x7fb8600cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8600cf710 0x7fb8600cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 -- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86015f7d0 msgr2=0x7fb8600b9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86015f7d0 0x7fb8600b9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 -- 172.21.15.96:0/360785802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb860157fb0 con 0x7fb8600d3470 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb857fff640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 0x7fb8600b8f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb84800b440 tx=0x7fb84800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848015020 con 0x7fb8600d3470 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb860158aa0 con 0x7fb8600d3470 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb848011980 con 0x7fb8600d3470 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848011dd0 con 0x7fb8600d3470 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8600ce690 con 0x7fb8600d3470 2024-09-21T02:10:21.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.607+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86015f290 con 0x7fb8600d3470 2024-09-21T02:10:21.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.611+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb848018020 con 0x7fb8600d3470 2024-09-21T02:10:21.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.611+0000 7fb856ffd640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb840073230 0x7fb840075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:21.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.611+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fb848094200 con 0x7fb8600d3470 2024-09-21T02:10:21.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.611+0000 7fb85e59c640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb840073230 0x7fb840075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:21.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.611+0000 7fb85e59c640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb840073230 0x7fb840075740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8500af000 tx=0x7fb85001b120 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:21.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.615+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb84801f030 con 0x7fb8600d3470 2024-09-21T02:10:21.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.831+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fb860050fd0 con 0x7fb8600d3470 2024-09-21T02:10:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb856ffd640 1 -- 172.21.15.96:0/360785802 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v26) ==== 90+0+0 (secure 0 0 0) 0x7fb84800d040 con 0x7fb8600d3470 2024-09-21T02:10:21.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb840073230 msgr2=0x7fb840075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb840073230 0x7fb840075740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8500af000 tx=0x7fb85001b120 comp rx=0 tx=0).stop 2024-09-21T02:10:21.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 msgr2=0x7fb8600b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 0x7fb8600b8f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb84800b440 tx=0x7fb84800b900 comp rx=0 tx=0).stop 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 shutdown_connections 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb840073230 0x7fb840075740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86015f7d0 0x7fb8600b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8600d3470 0x7fb8600b8f00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 --2- 172.21.15.96:0/360785802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8600cf710 0x7fb8600cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 >> 172.21.15.96:0/360785802 conn(0x7fb8600d1b10 msgr2=0x7fb860152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 shutdown_connections 2024-09-21T02:10:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:21.835+0000 7fb864d47640 1 -- 172.21.15.96:0/360785802 wait complete. 2024-09-21T02:10:21.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T02:10:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- 172.21.15.96:0/2288163470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae8155040 msgr2=0x7f3ae80ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 --2- 172.21.15.96:0/2288163470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae8155040 0x7f3ae80ca880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80734f0 tx=0x7f3ad801da00 comp rx=0 tx=0).stop 2024-09-21T02:10:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- 172.21.15.96:0/2288163470 shutdown_connections 2024-09-21T02:10:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 --2- 172.21.15.96:0/2288163470 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae806dea0 0x7f3ae806e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 --2- 172.21.15.96:0/2288163470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae8155040 0x7f3ae80ca880 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- 172.21.15.96:0/2288163470 >> 172.21.15.96:0/2288163470 conn(0x7f3ae806eea0 msgr2=0x7f3ae806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- 172.21.15.96:0/2288163470 shutdown_connections 2024-09-21T02:10:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- 172.21.15.96:0/2288163470 wait complete. 2024-09-21T02:10:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 Processor -- start 2024-09-21T02:10:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- start start 2024-09-21T02:10:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 0x7f3ae814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae8155040 0x7f3ae815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae815e3e0 0x7f3ae815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8168110 con 0x7f3ae806dea0 2024-09-21T02:10:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8167f90 con 0x7f3ae815e3e0 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3aee14c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae8168290 con 0x7f3ae8155040 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 0x7f3ae814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 0x7f3ae814a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57646/0 (socket says 172.21.15.96:57646) 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 -- 172.21.15.96:0/1396241081 learned_addr learned my addr 172.21.15.96:0/1396241081 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae7fff640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae815e3e0 0x7f3ae815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae6ffd640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae8155040 0x7f3ae815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 -- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae8155040 msgr2=0x7f3ae815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae8155040 0x7f3ae815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 -- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae815e3e0 msgr2=0x7f3ae815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae815e3e0 0x7f3ae815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae77fe640 1 -- 172.21.15.96:0/1396241081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80c67a0 con 0x7f3ae806dea0 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae7fff640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae815e3e0 0x7f3ae815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.007+0000 7f3ae6ffd640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae8155040 0x7f3ae815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3ae77fe640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 0x7f3ae814a9a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80b1040 tx=0x7f3ad80a6810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80b3040 con 0x7f3ae806dea0 2024-09-21T02:10:22.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad8003770 con 0x7f3ae806dea0 2024-09-21T02:10:22.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae80b3ac0 con 0x7f3ae806dea0 2024-09-21T02:10:22.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8003ba0 con 0x7f3ae806dea0 2024-09-21T02:10:22.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae80c5940 con 0x7f3ae806dea0 2024-09-21T02:10:22.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae806e2b0 con 0x7f3ae806dea0 2024-09-21T02:10:22.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ad80c0020 con 0x7f3ae806dea0 2024-09-21T02:10:22.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3acffff640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3ad4073200 0x7f3ad4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f3ad8130040 con 0x7f3ae806dea0 2024-09-21T02:10:22.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.011+0000 7f3ae6ffd640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3ad4073200 0x7f3ad4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.015+0000 7f3ae6ffd640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3ad4073200 0x7f3ad4075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ae81f8bb0 tx=0x7f3adc005cc0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.015+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3ad80c7020 con 0x7f3ae806dea0 2024-09-21T02:10:22.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.243+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f3ae80caba0 con 0x7f3ae806dea0 2024-09-21T02:10:22.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.251+0000 7f3acffff640 1 -- 172.21.15.96:0/1396241081 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v27) ==== 97+0+0 (secure 0 0 0) 0x7f3ae806e7f0 con 0x7f3ae806dea0 2024-09-21T02:10:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3ad4073200 msgr2=0x7f3ad4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3ad4073200 0x7f3ad4075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ae81f8bb0 tx=0x7f3adc005cc0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 msgr2=0x7f3ae814a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 0x7f3ae814a9a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80b1040 tx=0x7f3ad80a6810 comp rx=0 tx=0).stop 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 shutdown_connections 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3ad4073200 0x7f3ad4075710 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae815e3e0 0x7f3ae815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae8155040 0x7f3ae815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 --2- 172.21.15.96:0/1396241081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ae806dea0 0x7f3ae814a9a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 >> 172.21.15.96:0/1396241081 conn(0x7f3ae806eea0 msgr2=0x7f3ae8073770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 shutdown_connections 2024-09-21T02:10:22.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.255+0000 7f3aee14c640 1 -- 172.21.15.96:0/1396241081 wait complete. 2024-09-21T02:10:22.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T02:10:22.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- 172.21.15.96:0/3688638190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 msgr2=0x7f30fc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 --2- 172.21.15.96:0/3688638190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f30fc003e80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f310400c2e0 tx=0x7f310401cf80 comp rx=0 tx=0).stop 2024-09-21T02:10:22.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3113fff640 1 -- 172.21.15.96:0/3688638190 reap_dead start 2024-09-21T02:10:22.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- 172.21.15.96:0/3688638190 shutdown_connections 2024-09-21T02:10:22.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- 172.21.15.96:0/3688638190 >> 172.21.15.96:0/3688638190 conn(0x7f311406eea0 msgr2=0x7f311406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- 172.21.15.96:0/3688638190 shutdown_connections 2024-09-21T02:10:22.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- 172.21.15.96:0/3688638190 wait complete. 2024-09-21T02:10:22.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 Processor -- start 2024-09-21T02:10:22.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- start start 2024-09-21T02:10:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f31141fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f311413f4c0 0x7f31140b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3114157d20 0x7f3114158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3114167c50 con 0x7f30fc003a10 2024-09-21T02:10:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3114167ad0 con 0x7f311413f4c0 2024-09-21T02:10:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f3119dbd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3114167dd0 con 0x7f3114157d20 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f31137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f31141fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f31137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f31141fa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57664/0 (socket says 172.21.15.96:57664) 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.423+0000 7f31137fe640 1 -- 172.21.15.96:0/2085231562 learned_addr learned my addr 172.21.15.96:0/2085231562 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3112ffd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f311413f4c0 0x7f31140b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3113fff640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3114157d20 0x7f3114158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f31137fe640 1 -- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3114157d20 msgr2=0x7f3114158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f31137fe640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3114157d20 0x7f3114158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f31137fe640 1 -- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f311413f4c0 msgr2=0x7f31140b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f31137fe640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f311413f4c0 0x7f31140b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f31137fe640 1 -- 172.21.15.96:0/2085231562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31140b2ef0 con 0x7f30fc003a10 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f31137fe640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f31141fa3a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f310401ca40 tx=0x7f3104000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104006090 con 0x7f30fc003a10 2024-09-21T02:10:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3104006230 con 0x7f30fc003a10 2024-09-21T02:10:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104003620 con 0x7f30fc003a10 2024-09-21T02:10:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141fabf0 con 0x7f30fc003a10 2024-09-21T02:10:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141fb1b0 con 0x7f30fc003a10 2024-09-21T02:10:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31140cf010 con 0x7f30fc003a10 2024-09-21T02:10:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31040037c0 con 0x7f30fc003a10 2024-09-21T02:10:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f30fbfff640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3100073200 0x7f3100075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f310412e000 con 0x7f30fc003a10 2024-09-21T02:10:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3112ffd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3100073200 0x7f3100075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.427+0000 7f3112ffd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3100073200 0x7f3100075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f31140af740 tx=0x7f31080091e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.431+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3104016d30 con 0x7f30fc003a10 2024-09-21T02:10:22.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.671+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f3114073310 con 0x7f30fc003a10 2024-09-21T02:10:22.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.675+0000 7f30fbfff640 1 -- 172.21.15.96:0/2085231562 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7f31040f9270 con 0x7f30fc003a10 2024-09-21T02:10:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3100073200 msgr2=0x7f3100075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3100073200 0x7f3100075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f31140af740 tx=0x7f31080091e0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 msgr2=0x7f31141fa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f31141fa3a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f310401ca40 tx=0x7f3104000f40 comp rx=0 tx=0).stop 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 shutdown_connections 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3100073200 0x7f3100075710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3114157d20 0x7f3114158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f311413f4c0 0x7f31140b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 --2- 172.21.15.96:0/2085231562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30fc003a10 0x7f31141fa3a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 >> 172.21.15.96:0/2085231562 conn(0x7f311406eea0 msgr2=0x7f3114156200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 shutdown_connections 2024-09-21T02:10:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.679+0000 7f3119dbd640 1 -- 172.21.15.96:0/2085231562 wait complete. 2024-09-21T02:10:22.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:10:22.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 -- 172.21.15.96:0/1440757157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba0c0051b0 msgr2=0x7fba0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 --2- 172.21.15.96:0/1440757157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba0c0051b0 0x7fba0c00b9d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fba14009ee0 tx=0x7fba1401d630 comp rx=0 tx=0).stop 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba2359e640 1 -- 172.21.15.96:0/1440757157 reap_dead start 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 -- 172.21.15.96:0/1440757157 shutdown_connections 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 -- 172.21.15.96:0/1440757157 >> 172.21.15.96:0/1440757157 conn(0x7fba2406eea0 msgr2=0x7fba2406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 -- 172.21.15.96:0/1440757157 shutdown_connections 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 -- 172.21.15.96:0/1440757157 wait complete. 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 Processor -- start 2024-09-21T02:10:22.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.855+0000 7fba291f6640 1 -- start start 2024-09-21T02:10:22.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba0c0051b0 0x7fba240c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 0x7fba241e9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba241e9870 0x7fba2415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba2415cbb0 con 0x7fba24158270 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba2415ca30 con 0x7fba241e9870 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fba240cf010 con 0x7fba0c0051b0 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 0x7fba241e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 0x7fba241e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57674/0 (socket says 172.21.15.96:57674) 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 -- 172.21.15.96:0/4277461773 learned_addr learned my addr 172.21.15.96:0/4277461773 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2359e640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba241e9870 0x7fba2415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba22d9d640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba0c0051b0 0x7fba240c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 -- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba0c0051b0 msgr2=0x7fba240c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba0c0051b0 0x7fba240c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 -- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba241e9870 msgr2=0x7fba2415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba241e9870 0x7fba2415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 -- 172.21.15.96:0/4277461773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba241e9c80 con 0x7fba24158270 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2359e640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba241e9870 0x7fba2415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba2259c640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 0x7fba241e9330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fba240b6130 tx=0x7fba1800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18004230 con 0x7fba24158270 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba180044c0 con 0x7fba24158270 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1800f8f0 con 0x7fba24158270 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba240af470 con 0x7fba24158270 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba240c84a0 con 0x7fba24158270 2024-09-21T02:10:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.859+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba240af1e0 con 0x7fba24158270 2024-09-21T02:10:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.863+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba1800fa90 con 0x7fba24158270 2024-09-21T02:10:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.863+0000 7fba0b7fe640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fba10073230 0x7fba10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.863+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fba18094080 con 0x7fba24158270 2024-09-21T02:10:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.863+0000 7fba22d9d640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fba10073230 0x7fba10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.863+0000 7fba22d9d640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fba10073230 0x7fba10075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0036b0 tx=0x7fba1c00c040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:22.867+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fba18020070 con 0x7fba24158270 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.043+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fba240c6220 con 0x7fba24158270 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.043+0000 7fba0b7fe640 1 -- 172.21.15.96:0/4277461773 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7fba241e9c80 con 0x7fba24158270 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-21T02:10:23.049 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-21T02:10:23.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:23.050 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fba10073230 msgr2=0x7fba10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fba10073230 0x7fba10075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0036b0 tx=0x7fba1c00c040 comp rx=0 tx=0).stop 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 msgr2=0x7fba241e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 0x7fba241e9330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fba240b6130 tx=0x7fba1800da20 comp rx=0 tx=0).stop 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 shutdown_connections 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fba10073230 0x7fba10075740 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba241e9870 0x7fba2415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba24158270 0x7fba241e9330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 --2- 172.21.15.96:0/4277461773 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba0c0051b0 0x7fba240c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 >> 172.21.15.96:0/4277461773 conn(0x7fba2406eea0 msgr2=0x7fba240b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 shutdown_connections 2024-09-21T02:10:23.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.047+0000 7fba291f6640 1 -- 172.21.15.96:0/4277461773 wait complete. 2024-09-21T02:10:23.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:10:23.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.219+0000 7f1a499c0640 1 -- 172.21.15.96:0/1121391272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a44073370 msgr2=0x7f1a44073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.219+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1121391272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a44073370 0x7f1a44073780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c80 tx=0x7f1a3c01cc80 comp rx=0 tx=0).stop 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.219+0000 7f1a499c0640 1 -- 172.21.15.96:0/1121391272 shutdown_connections 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.219+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1121391272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a44073370 0x7f1a44073780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.219+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1121391272 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a4415f770 0x7f1a440b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.219+0000 7f1a499c0640 1 -- 172.21.15.96:0/1121391272 >> 172.21.15.96:0/1121391272 conn(0x7f1a440d05a0 msgr2=0x7f1a440d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 -- 172.21.15.96:0/1121391272 shutdown_connections 2024-09-21T02:10:23.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 -- 172.21.15.96:0/1121391272 wait complete. 2024-09-21T02:10:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 Processor -- start 2024-09-21T02:10:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 -- start start 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a44073370 0x7f1a440adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a440b2ef0 0x7f1a44148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 0x7f1a4414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44168c60 con 0x7f1a441499e0 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44168ae0 con 0x7f1a440b2ef0 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a499c0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1a44168de0 con 0x7f1a44073370 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 0x7f1a4414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 0x7f1a4414d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57692/0 (socket says 172.21.15.96:57692) 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 -- 172.21.15.96:0/1782851506 learned_addr learned my addr 172.21.15.96:0/1782851506 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:23.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a427fc640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a440b2ef0 0x7f1a44148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a42ffd640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a44073370 0x7f1a440adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 -- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a44073370 msgr2=0x7f1a440adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a44073370 0x7f1a440adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 -- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a440b2ef0 msgr2=0x7f1a44148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a440b2ef0 0x7f1a44148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 -- 172.21.15.96:0/1782851506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a441f6e20 con 0x7f1a441499e0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a42ffd640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a44073370 0x7f1a440adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.223+0000 7f1a437fe640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 0x7f1a4414d820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c01cab0 tx=0x7f1a3c0a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0bb070 con 0x7f1a441499e0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a3c004d60 con 0x7f1a441499e0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a44070060 con 0x7f1a441499e0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0b2040 con 0x7f1a441499e0 2024-09-21T02:10:23.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a44149210 con 0x7f1a441499e0 2024-09-21T02:10:23.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44073780 con 0x7f1a441499e0 2024-09-21T02:10:23.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1a3c0be070 con 0x7f1a441499e0 2024-09-21T02:10:23.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a2bfff640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1a30073200 0x7f1a30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1a3c12e170 con 0x7f1a441499e0 2024-09-21T02:10:23.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a42ffd640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1a30073200 0x7f1a30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.227+0000 7f1a42ffd640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1a30073200 0x7f1a30075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400a850 tx=0x7f1a34008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.231+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a3c016990 con 0x7f1a441499e0 2024-09-21T02:10:23.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:23.391+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1a44150820 con 0x7f1a441499e0 2024-09-21T02:10:24.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.255+0000 7f1a2bfff640 1 -- 172.21.15.96:0/1782851506 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f1a3c0f94c0 con 0x7f1a441499e0 2024-09-21T02:10:24.261 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1a30073200 msgr2=0x7f1a30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1a30073200 0x7f1a30075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400a850 tx=0x7f1a34008040 comp rx=0 tx=0).stop 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 msgr2=0x7f1a4414d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 0x7f1a4414d820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c01cab0 tx=0x7f1a3c0a5a00 comp rx=0 tx=0).stop 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 shutdown_connections 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1a30073200 0x7f1a30075710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a441499e0 0x7f1a4414d820 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a440b2ef0 0x7f1a44148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 --2- 172.21.15.96:0/1782851506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a44073370 0x7f1a440adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 >> 172.21.15.96:0/1782851506 conn(0x7f1a440d05a0 msgr2=0x7f1a4414c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 shutdown_connections 2024-09-21T02:10:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.259+0000 7f1a499c0640 1 -- 172.21.15.96:0/1782851506 wait complete. 2024-09-21T02:10:24.297 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 -- 172.21.15.96:0/2115872471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec15ece0 msgr2=0x7f56ec1512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 --2- 172.21.15.96:0/2115872471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec15ece0 0x7f56ec1512f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009840 tx=0x7f56dc01c820 comp rx=0 tx=0).stop 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 -- 172.21.15.96:0/2115872471 shutdown_connections 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 --2- 172.21.15.96:0/2115872471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ec151830 0x7f56ec06f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 --2- 172.21.15.96:0/2115872471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec15ece0 0x7f56ec1512f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 -- 172.21.15.96:0/2115872471 >> 172.21.15.96:0/2115872471 conn(0x7f56ec0d0490 msgr2=0x7f56ec06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 -- 172.21.15.96:0/2115872471 shutdown_connections 2024-09-21T02:10:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 -- 172.21.15.96:0/2115872471 wait complete. 2024-09-21T02:10:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 Processor -- start 2024-09-21T02:10:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.459+0000 7f56f3347640 1 -- start start 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 0x7f56ec14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ec151830 0x7f56ec14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56ec14f230 0x7f56ec14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec070010 con 0x7f56ec0cf680 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec06fe90 con 0x7f56ec14f230 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f56ec070190 con 0x7f56ec151830 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 0x7f56ec14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 0x7f56ec14eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57716/0 (socket says 172.21.15.96:57716) 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 -- 172.21.15.96:0/3950245202 learned_addr learned my addr 172.21.15.96:0/3950245202 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f18e4640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56ec14f230 0x7f56ec14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 -- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ec151830 msgr2=0x7f56ec14b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ec151830 0x7f56ec14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 -- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56ec14f230 msgr2=0x7f56ec14f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56ec14f230 0x7f56ec14f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 -- 172.21.15.96:0/3950245202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec14fb80 con 0x7f56ec0cf680 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f10e3640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 0x7f56ec14eb90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c400 tx=0x7f56e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8011790 con 0x7f56ec0cf680 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56e8017070 con 0x7f56ec0cf680 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec156450 con 0x7f56ec0cf680 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8011cc0 con 0x7f56ec0cf680 2024-09-21T02:10:24.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec14d630 con 0x7f56ec0cf680 2024-09-21T02:10:24.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec06f740 con 0x7f56ec0cf680 2024-09-21T02:10:24.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.463+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f56e801c460 con 0x7f56ec0cf680 2024-09-21T02:10:24.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.467+0000 7f56e1ffb640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f56d0073230 0x7f56d0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:24.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.467+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f56e8093fa0 con 0x7f56ec0cf680 2024-09-21T02:10:24.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.467+0000 7f56f08e2640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f56d0073230 0x7f56d0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:24.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.467+0000 7f56f08e2640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f56d0073230 0x7f56d0075740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0af000 tx=0x7f56dc0195c0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:24.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.471+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f56ec14fb80 con 0x7f56ec0cf680 2024-09-21T02:10:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:24.647+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f56ec1512d0 con 0x7f56ec0cf680 2024-09-21T02:10:25.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.271+0000 7f56e1ffb640 1 -- 172.21.15.96:0/3950245202 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f56e805f010 con 0x7f56ec0cf680 2024-09-21T02:10:25.276 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f56d0073230 msgr2=0x7f56d0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f56d0073230 0x7f56d0075740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0af000 tx=0x7f56dc0195c0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 msgr2=0x7f56ec14eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 0x7f56ec14eb90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c400 tx=0x7f56e800c8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 shutdown_connections 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f56d0073230 0x7f56d0075740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56ec14f230 0x7f56ec14f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ec151830 0x7f56ec14b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 --2- 172.21.15.96:0/3950245202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec0cf680 0x7f56ec14eb90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 >> 172.21.15.96:0/3950245202 conn(0x7f56ec0d0490 msgr2=0x7f56ec0cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 shutdown_connections 2024-09-21T02:10:25.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.275+0000 7f56f3347640 1 -- 172.21.15.96:0/3950245202 wait complete. 2024-09-21T02:10:25.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 -- 172.21.15.96:0/3032988903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 msgr2=0x7f7ddc15ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 --2- 172.21.15.96:0/3032988903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc15ef70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009900 tx=0x7f7dcc01c5e0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 -- 172.21.15.96:0/3032988903 shutdown_connections 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 --2- 172.21.15.96:0/3032988903 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ddc0cfed0 0x7f7ddc073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 --2- 172.21.15.96:0/3032988903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc15ef70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 -- 172.21.15.96:0/3032988903 >> 172.21.15.96:0/3032988903 conn(0x7f7ddc0d13d0 msgr2=0x7f7ddc0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 -- 172.21.15.96:0/3032988903 shutdown_connections 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.479+0000 7f7de1a92640 1 -- 172.21.15.96:0/3032988903 wait complete. 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 Processor -- start 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 -- start start 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ddc0cfed0 0x7f7ddc149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ddc15f4b0 0x7f7ddc0b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc169110 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc070120 con 0x7f7ddc0cfed0 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7ddc169290 con 0x7f7ddc15f4b0 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57738/0 (socket says 172.21.15.96:57738) 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 -- 172.21.15.96:0/1571192339 learned_addr learned my addr 172.21.15.96:0/1571192339 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7dd2ffd640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ddc0cfed0 0x7f7ddc149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddbfff640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ddc15f4b0 0x7f7ddc0b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 -- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ddc15f4b0 msgr2=0x7f7ddc0b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ddc15f4b0 0x7f7ddc0b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 -- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ddc0cfed0 msgr2=0x7f7ddc149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ddc0cfed0 0x7f7ddc149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 -- 172.21.15.96:0/1571192339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc146fb0 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7ddb7fe640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc146a70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009850 tx=0x7f7dcc0ab580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0aba60 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dcc0abcf0 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc1fa3f0 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc003860 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.483+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc1fabd0 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.487+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc15ef70 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.487+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7dcc0bd020 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.487+0000 7f7dd8ff9640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7dbc073230 0x7f7dbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.487+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f7dcc12f4c0 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.487+0000 7f7dd2ffd640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7dbc073230 0x7f7dbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.487+0000 7f7dd2ffd640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7dbc073230 0x7f7dbc075740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc1539e0 tx=0x7f7dc4005db0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.491+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7ddc15b980 con 0x7f7ddc0cfac0 2024-09-21T02:10:25.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:25.707+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7ddc153cf0 con 0x7f7ddc0cfac0 2024-09-21T02:10:26.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.271+0000 7f7dd8ff9640 1 -- 172.21.15.96:0/1571192339 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f7dcc0fa520 con 0x7f7ddc0cfac0 2024-09-21T02:10:26.278 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 5 and data pool 6 2024-09-21T02:10:26.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.279+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7dbc073230 msgr2=0x7f7dbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.279+0000 7f7de1a92640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7dbc073230 0x7f7dbc075740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc1539e0 tx=0x7f7dc4005db0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.279+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 msgr2=0x7f7ddc146a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.279+0000 7f7de1a92640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc146a70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009850 tx=0x7f7dcc0ab580 comp rx=0 tx=0).stop 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 shutdown_connections 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7dbc073230 0x7f7dbc075740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ddc15f4b0 0x7f7ddc0b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ddc0cfed0 0x7f7ddc149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 --2- 172.21.15.96:0/1571192339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc0cfac0 0x7f7ddc146a70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 >> 172.21.15.96:0/1571192339 conn(0x7f7ddc0d13d0 msgr2=0x7f7ddc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.283+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 shutdown_connections 2024-09-21T02:10:26.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.291+0000 7f7de1a92640 1 -- 172.21.15.96:0/1571192339 wait complete. 2024-09-21T02:10:26.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:10:26.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 -- 172.21.15.96:0/2640498924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc806e3e0 msgr2=0x7fecc815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 --2- 172.21.15.96:0/2640498924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc806e3e0 0x7fecc815bea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fecc806a1c0 tx=0x7fecbc01c6e0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 -- 172.21.15.96:0/2640498924 shutdown_connections 2024-09-21T02:10:26.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 --2- 172.21.15.96:0/2640498924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc806e3e0 0x7fecc815bea0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 --2- 172.21.15.96:0/2640498924 >> v2:172.21.15.122:3300/0 conn(0x7fecc80ced00 0x7fecc806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 -- 172.21.15.96:0/2640498924 >> 172.21.15.96:0/2640498924 conn(0x7fecc806eea0 msgr2=0x7fecc806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 -- 172.21.15.96:0/2640498924 shutdown_connections 2024-09-21T02:10:26.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.479+0000 7feccf8bb640 1 -- 172.21.15.96:0/2640498924 wait complete. 2024-09-21T02:10:26.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 Processor -- start 2024-09-21T02:10:26.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 -- start start 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecc806e3e0 0x7fecc81509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fecc80ced00 0x7fecc815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 0x7fecc815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8168bf0 con 0x7fecc815c3e0 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fecc80cf300 con 0x7fecc806e3e0 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccf8bb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fecc8168d70 con 0x7fecc80ced00 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 0x7fecc815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 0x7fecc815adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57758/0 (socket says 172.21.15.96:57758) 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 -- 172.21.15.96:0/4204434948 learned_addr learned my addr 172.21.15.96:0/4204434948 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccd657640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecc806e3e0 0x7fecc81509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccce56640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fecc80ced00 0x7fecc815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 -- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fecc80ced00 msgr2=0x7fecc815a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fecc80ced00 0x7fecc815a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 -- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecc806e3e0 msgr2=0x7fecc81509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecc806e3e0 0x7fecc81509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 -- 172.21.15.96:0/4204434948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc8151320 con 0x7fecc815c3e0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccce56640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fecc80ced00 0x7fecc815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7feccde58640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 0x7fecc815adf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fecb800b4c0 tx=0x7fecb800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.483+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8015020 con 0x7fecc815c3e0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb8011920 con 0x7fecc815c3e0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc8151600 con 0x7fecc815c3e0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8011d70 con 0x7fecc815c3e0 2024-09-21T02:10:26.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc8165ca0 con 0x7fecc815c3e0 2024-09-21T02:10:26.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc81572e0 con 0x7fecc815c3e0 2024-09-21T02:10:26.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fecb8018020 con 0x7fecc815c3e0 2024-09-21T02:10:26.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7fecc5ffb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fecb4073200 0x7fecb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fecb8093f60 con 0x7fecc815c3e0 2024-09-21T02:10:26.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7feccd657640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fecb4073200 0x7fecb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.487+0000 7feccd657640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fecb4073200 0x7fecb4075710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecc0005c50 tx=0x7fecc000a390 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.491+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fecc81572e0 con 0x7fecc815c3e0 2024-09-21T02:10:26.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.667+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecc815bea0 con 0x7fecc815c3e0 2024-09-21T02:10:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.667+0000 7fecc5ffb640 1 -- 172.21.15.96:0/4204434948 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17015 (secure 0 0 0) 0x7fecb801f030 con 0x7fecc815c3e0 2024-09-21T02:10:26.675 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:26.675 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":39,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:10:25.710429+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:10:23.405387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T02:10:24.661586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fecb4073200 msgr2=0x7fecb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fecb4073200 0x7fecb4075710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecc0005c50 tx=0x7fecc000a390 comp rx=0 tx=0).stop 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 msgr2=0x7fecc815adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 0x7fecc815adf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fecb800b4c0 tx=0x7fecb800b980 comp rx=0 tx=0).stop 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 shutdown_connections 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fecb4073200 0x7fecb4075710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecc815c3e0 0x7fecc815adf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fecc80ced00 0x7fecc815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 --2- 172.21.15.96:0/4204434948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecc806e3e0 0x7fecc81509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 >> 172.21.15.96:0/4204434948 conn(0x7fecc806eea0 msgr2=0x7fecc80ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 shutdown_connections 2024-09-21T02:10:26.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.671+0000 7feccf8bb640 1 -- 172.21.15.96:0/4204434948 wait complete. 2024-09-21T02:10:26.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:10:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 -- 172.21.15.96:0/464613053 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c15f7d0 msgr2=0x7f703c162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 --2- 172.21.15.96:0/464613053 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c15f7d0 0x7f703c162410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f702c009900 tx=0x7f702c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 -- 172.21.15.96:0/464613053 shutdown_connections 2024-09-21T02:10:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 --2- 172.21.15.96:0/464613053 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c15f7d0 0x7f703c162410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 --2- 172.21.15.96:0/464613053 >> v2:172.21.15.122:3300/0 conn(0x7f703c0cf710 0x7f703c15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 -- 172.21.15.96:0/464613053 >> 172.21.15.96:0/464613053 conn(0x7f703c0d1b10 msgr2=0x7f703c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 -- 172.21.15.96:0/464613053 shutdown_connections 2024-09-21T02:10:26.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.839+0000 7f7043d0b640 1 -- 172.21.15.96:0/464613053 wait complete. 2024-09-21T02:10:26.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 Processor -- start 2024-09-21T02:10:26.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 -- start start 2024-09-21T02:10:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 0x7f703c1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f703c15f7d0 0x7f703c1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c162950 0x7f703c1486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0d3500 con 0x7f703c0cf710 2024-09-21T02:10:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0d3380 con 0x7f703c15f7d0 2024-09-21T02:10:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7043d0b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f703c0d3680 con 0x7f703c162950 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 0x7f703c1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 0x7f703c1522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57766/0 (socket says 172.21.15.96:57766) 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 -- 172.21.15.96:0/159882568 learned_addr learned my addr 172.21.15.96:0/159882568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f70422a8640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c162950 0x7f703c1486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f70412a6640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f703c15f7d0 0x7f703c1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 -- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c162950 msgr2=0x7f703c1486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c162950 0x7f703c1486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 -- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f703c15f7d0 msgr2=0x7f703c1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f703c15f7d0 0x7f703c1585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f7041aa7640 1 -- 172.21.15.96:0/159882568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c153390 con 0x7f703c0cf710 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.843+0000 7f70412a6640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f703c15f7d0 0x7f703c1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f7041aa7640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 0x7f703c1522e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f703800e770 tx=0x7f703800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800ccc0 con 0x7f703c0cf710 2024-09-21T02:10:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f703800ee80 con 0x7f703c0cf710 2024-09-21T02:10:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c153670 con 0x7f703c0cf710 2024-09-21T02:10:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800ce60 con 0x7f703c0cf710 2024-09-21T02:10:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c152820 con 0x7f703c0cf710 2024-09-21T02:10:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c15f290 con 0x7f703c0cf710 2024-09-21T02:10:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7038004260 con 0x7f703c0cf710 2024-09-21T02:10:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70327fc640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7020073200 0x7f7020075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7038094d10 con 0x7f703c0cf710 2024-09-21T02:10:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70412a6640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7020073200 0x7f7020075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.847+0000 7f70412a6640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7020073200 0x7f7020075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f702c0af000 tx=0x7f702c01b120 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:26.851+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f703805fc30 con 0x7f703c0cf710 2024-09-21T02:10:27.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.011+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f703c14c910 con 0x7f703c0cf710 2024-09-21T02:10:27.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.011+0000 7f70327fc640 1 -- 172.21.15.96:0/159882568 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17015 (secure 0 0 0) 0x7f703805f530 con 0x7f703c0cf710 2024-09-21T02:10:27.019 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:27.019 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":39,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:10:25.710429+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:10:23.405387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T02:10:24.661586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:10:27.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7020073200 msgr2=0x7f7020075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:27.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7020073200 0x7f7020075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f702c0af000 tx=0x7f702c01b120 comp rx=0 tx=0).stop 2024-09-21T02:10:27.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 msgr2=0x7f703c1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 0x7f703c1522e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f703800e770 tx=0x7f703800ec30 comp rx=0 tx=0).stop 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 shutdown_connections 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7020073200 0x7f7020075710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f703c162950 0x7f703c1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f703c15f7d0 0x7f703c1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 --2- 172.21.15.96:0/159882568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c0cf710 0x7f703c1522e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 >> 172.21.15.96:0/159882568 conn(0x7f703c0d1b10 msgr2=0x7f703c1618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 shutdown_connections 2024-09-21T02:10:27.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.015+0000 7f7043d0b640 1 -- 172.21.15.96:0/159882568 wait complete. 2024-09-21T02:10:27.039 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:10:27.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 -- 172.21.15.96:0/1053533042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339006dea0 msgr2=0x7f339006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:27.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 --2- 172.21.15.96:0/1053533042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339006dea0 0x7f339006e2b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f33900cdb80 tx=0x7f338001d9e0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 -- 172.21.15.96:0/1053533042 shutdown_connections 2024-09-21T02:10:27.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 --2- 172.21.15.96:0/1053533042 >> v2:172.21.15.122:3300/0 conn(0x7f33900cab60 0x7f33900caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 --2- 172.21.15.96:0/1053533042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339006dea0 0x7f339006e2b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 -- 172.21.15.96:0/1053533042 >> 172.21.15.96:0/1053533042 conn(0x7f339006eea0 msgr2=0x7f339006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:27.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 -- 172.21.15.96:0/1053533042 shutdown_connections 2024-09-21T02:10:27.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 -- 172.21.15.96:0/1053533042 wait complete. 2024-09-21T02:10:27.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 Processor -- start 2024-09-21T02:10:27.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.179+0000 7f3397d05640 1 -- start start 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f339006dea0 0x7f33900c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33900cab60 0x7f33900c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 0x7f33900c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33900cbda0 con 0x7f3390155240 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33900cbc20 con 0x7f33900cab60 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f33900cbf20 con 0x7f339006dea0 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 0x7f33900c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 0x7f33900c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46908/0 (socket says 172.21.15.96:46908) 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 -- 172.21.15.96:0/3957974665 learned_addr learned my addr 172.21.15.96:0/3957974665 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3395aa1640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f339006dea0 0x7f33900c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33952a0640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33900cab60 0x7f33900c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 -- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f339006dea0 msgr2=0x7f33900c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f339006dea0 0x7f33900c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 -- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33900cab60 msgr2=0x7f33900c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33900cab60 0x7f33900c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 -- 172.21.15.96:0/3957974665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900c7160 con 0x7f3390155240 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33962a2640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 0x7f33900c3b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f338800ca90 tx=0x7f338800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388007df0 con 0x7f3390155240 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3388004b30 con 0x7f3390155240 2024-09-21T02:10:27.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388005020 con 0x7f3390155240 2024-09-21T02:10:27.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33900c6590 con 0x7f3390155240 2024-09-21T02:10:27.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33900c7350 con 0x7f3390155240 2024-09-21T02:10:27.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339006e2b0 con 0x7f3390155240 2024-09-21T02:10:27.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33880051c0 con 0x7f3390155240 2024-09-21T02:10:27.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33867fc640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3374073200 0x7f3374075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:27.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f3388094a30 con 0x7f3390155240 2024-09-21T02:10:27.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.183+0000 7f3395aa1640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3374073200 0x7f3374075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:27.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.187+0000 7f3395aa1640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3374073200 0x7f3374075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f338c00b810 tx=0x7f338c008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:27.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.187+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3388014030 con 0x7f3390155240 2024-09-21T02:10:27.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:27.387+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f339006e9d0 con 0x7f3390155240 2024-09-21T02:10:28.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.319+0000 7f33867fc640 1 -- 172.21.15.96:0/3957974665 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f338805f9e0 con 0x7f3390155240 2024-09-21T02:10:28.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3374073200 msgr2=0x7f3374075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3374073200 0x7f3374075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f338c00b810 tx=0x7f338c008040 comp rx=0 tx=0).stop 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 msgr2=0x7f33900c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 0x7f33900c3b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f338800ca90 tx=0x7f338800cf50 comp rx=0 tx=0).stop 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 shutdown_connections 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3374073200 0x7f3374075710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3390155240 0x7f33900c3b50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33900cab60 0x7f33900c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 --2- 172.21.15.96:0/3957974665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f339006dea0 0x7f33900c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 >> 172.21.15.96:0/3957974665 conn(0x7f339006eea0 msgr2=0x7f33900733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 shutdown_connections 2024-09-21T02:10:28.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.323+0000 7f3397d05640 1 -- 172.21.15.96:0/3957974665 wait complete. 2024-09-21T02:10:28.349 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:10:28.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:10:28.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 -- 172.21.15.96:0/4188350293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 msgr2=0x7f79dc0c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/4188350293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc0c4ed0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f79dc06a1c0 tx=0x7f79cc01c770 comp rx=0 tx=0).stop 2024-09-21T02:10:28.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 -- 172.21.15.96:0/4188350293 shutdown_connections 2024-09-21T02:10:28.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/4188350293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc0c4ed0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/4188350293 >> v2:172.21.15.122:3300/0 conn(0x7f79dc06e2c0 0x7f79dc0c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 -- 172.21.15.96:0/4188350293 >> 172.21.15.96:0/4188350293 conn(0x7f79dc06eea0 msgr2=0x7f79dc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 -- 172.21.15.96:0/4188350293 shutdown_connections 2024-09-21T02:10:28.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.503+0000 7f79e3ffc640 1 -- 172.21.15.96:0/4188350293 wait complete. 2024-09-21T02:10:28.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 Processor -- start 2024-09-21T02:10:28.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 -- start start 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79dc06e2c0 0x7f79dc1f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc14f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79dc0cdbf0 0x7f79dc158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc0cf0a0 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc0cef20 con 0x7f79dc0cdbf0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc0cf220 con 0x7f79dc06e2c0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc14f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc14f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46932/0 (socket says 172.21.15.96:46932) 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 -- 172.21.15.96:0/3404847216 learned_addr learned my addr 172.21.15.96:0/3404847216 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e2599640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79dc0cdbf0 0x7f79dc158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1d98640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79dc06e2c0 0x7f79dc1f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 -- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79dc06e2c0 msgr2=0x7f79dc1f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79dc06e2c0 0x7f79dc1f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 -- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79dc0cdbf0 msgr2=0x7f79dc158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79dc0cdbf0 0x7f79dc158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 -- 172.21.15.96:0/3404847216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc14fa00 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1d98640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79dc06e2c0 0x7f79dc1f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e1597640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc14f050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f79cc000c10 tx=0x7f79cc0a6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc004030 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79cc0041d0 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79dc14fce0 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0050c0 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.507+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79dc1502a0 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.511+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79dc159d10 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.511+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f79cc0b8030 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.511+0000 7f79d27fc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f79c0073200 0x7f79c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.511+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f79cc12e5a0 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.511+0000 7f79e1d98640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f79c0073200 0x7f79c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.511+0000 7f79e1d98640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f79c0073200 0x7f79c0075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009d90 tx=0x7f79d4009320 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.515+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f79cc0bf070 con 0x7f79dc0c4ac0 2024-09-21T02:10:28.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:28.727+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f79dc1f7a20 con 0x7f79dc0c4ac0 2024-09-21T02:10:29.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.327+0000 7f79d27fc640 1 -- 172.21.15.96:0/3404847216 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f79cc0ac030 con 0x7f79dc0c4ac0 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f79c0073200 msgr2=0x7f79c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f79c0073200 0x7f79c0075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009d90 tx=0x7f79d4009320 comp rx=0 tx=0).stop 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 msgr2=0x7f79dc14f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc14f050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f79cc000c10 tx=0x7f79cc0a6670 comp rx=0 tx=0).stop 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 shutdown_connections 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f79c0073200 0x7f79c0075710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79dc0cdbf0 0x7f79dc158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79dc0c4ac0 0x7f79dc14f050 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 --2- 172.21.15.96:0/3404847216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79dc06e2c0 0x7f79dc1f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 >> 172.21.15.96:0/3404847216 conn(0x7f79dc06eea0 msgr2=0x7f79dc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 shutdown_connections 2024-09-21T02:10:29.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.331+0000 7f79e3ffc640 1 -- 172.21.15.96:0/3404847216 wait complete. 2024-09-21T02:10:29.362 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:10:29.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:10:29.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 -- 172.21.15.96:0/2627456957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c0cf710 msgr2=0x7f461c15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 --2- 172.21.15.96:0/2627456957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c0cf710 0x7f461c15f290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f461c06ecf0 tx=0x7f461001c8a0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 -- 172.21.15.96:0/2627456957 shutdown_connections 2024-09-21T02:10:29.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 --2- 172.21.15.96:0/2627456957 >> v2:172.21.15.122:3300/0 conn(0x7f461c15f7d0 0x7f461c162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 --2- 172.21.15.96:0/2627456957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c0cf710 0x7f461c15f290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 -- 172.21.15.96:0/2627456957 >> 172.21.15.96:0/2627456957 conn(0x7f461c0d1b10 msgr2=0x7f461c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:29.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 -- 172.21.15.96:0/2627456957 shutdown_connections 2024-09-21T02:10:29.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.519+0000 7f46227a0640 1 -- 172.21.15.96:0/2627456957 wait complete. 2024-09-21T02:10:29.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 Processor -- start 2024-09-21T02:10:29.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 -- start start 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f461c0cf710 0x7f461c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 0x7f461c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f461c162950 0x7f461c1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f461c0d3500 con 0x7f461c15f7d0 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f461c0d3380 con 0x7f461c162950 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f461c0d3680 con 0x7f461c0cf710 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 0x7f461c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 0x7f461c150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46952/0 (socket says 172.21.15.96:46952) 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 -- 172.21.15.96:0/78073756 learned_addr learned my addr 172.21.15.96:0/78073756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461bfff640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f461c0cf710 0x7f461c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f4620d3d640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f461c162950 0x7f461c1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 -- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f461c0cf710 msgr2=0x7f461c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f461c0cf710 0x7f461c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 -- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f461c162950 msgr2=0x7f461c1514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f461c162950 0x7f461c1514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 -- 172.21.15.96:0/78073756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c0b3cd0 con 0x7f461c15f7d0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f4620d3d640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f461c162950 0x7f461c1514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f461b7fe640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 0x7f461c150f90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c9f0 tx=0x7f460c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c007bb0 con 0x7f461c15f7d0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f460c007e40 con 0x7f461c15f7d0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c0b41d0 con 0x7f461c15f7d0 2024-09-21T02:10:29.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c00f690 con 0x7f461c15f7d0 2024-09-21T02:10:29.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.523+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c0b4790 con 0x7f461c15f7d0 2024-09-21T02:10:29.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.527+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c0b3f40 con 0x7f461c15f7d0 2024-09-21T02:10:29.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.527+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f460c00f830 con 0x7f461c15f7d0 2024-09-21T02:10:29.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.527+0000 7f4618ff9640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4608073200 0x7f4608075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.527+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f460c094b60 con 0x7f461c15f7d0 2024-09-21T02:10:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.527+0000 7f461bfff640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4608073200 0x7f4608075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.527+0000 7f461bfff640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4608073200 0x7f4608075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f46100af000 tx=0x7f461001b0e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:29.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.531+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f460c014030 con 0x7f461c15f7d0 2024-09-21T02:10:29.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.743+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f461c15f290 con 0x7f461c15f7d0 2024-09-21T02:10:29.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.743+0000 7f4618ff9640 1 -- 172.21.15.96:0/78073756 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f460c05fa70 con 0x7f461c15f7d0 2024-09-21T02:10:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:29.750 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"btime":"2024-09-21T02:10:29:323484+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:10:25.710385+0000","modified":"2024-09-21T02:10:28.731053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5037":{"gid":5037,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3851681791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3851681791},{"type":"v1","addr":"172.21.15.96:6835","nonce":3851681791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5037,"qdb_cluster":[5037]},"id":2}]} 2024-09-21T02:10:29.751 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-09-21T02:10:29.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4608073200 msgr2=0x7f4608075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4608073200 0x7f4608075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f46100af000 tx=0x7f461001b0e0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 msgr2=0x7f461c150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 0x7f461c150f90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c9f0 tx=0x7f460c00ceb0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 shutdown_connections 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4608073200 0x7f4608075710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f461c162950 0x7f461c1514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f461c15f7d0 0x7f461c150f90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 --2- 172.21.15.96:0/78073756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f461c0cf710 0x7f461c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 >> 172.21.15.96:0/78073756 conn(0x7f461c0d1b10 msgr2=0x7f461c0b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 shutdown_connections 2024-09-21T02:10:29.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.747+0000 7f46227a0640 1 -- 172.21.15.96:0/78073756 wait complete. 2024-09-21T02:10:29.757 INFO:tasks.ceph.mds.a.smithi096.stderr:2024-09-21T02:10:29.751+0000 7f821d9f8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:10:29.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 -- 172.21.15.96:0/672145797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c406e2c0 msgr2=0x7f30c40c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/672145797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c406e2c0 0x7f30c40c4580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f30c40727f0 tx=0x7f30b401c850 comp rx=0 tx=0).stop 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 -- 172.21.15.96:0/672145797 shutdown_connections 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/672145797 >> v2:172.21.15.122:3300/0 conn(0x7f30c40cdbf0 0x7f30c40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/672145797 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30c40c4ac0 0x7f30c40c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/672145797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c406e2c0 0x7f30c40c4580 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 -- 172.21.15.96:0/672145797 >> 172.21.15.96:0/672145797 conn(0x7f30c406eea0 msgr2=0x7f30c406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 -- 172.21.15.96:0/672145797 shutdown_connections 2024-09-21T02:10:29.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 -- 172.21.15.96:0/672145797 wait complete. 2024-09-21T02:10:29.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 Processor -- start 2024-09-21T02:10:29.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.915+0000 7f30ca1c7640 1 -- start start 2024-09-21T02:10:29.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30c406e2c0 0x7f30c415fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 0x7f30c40c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30c40cdbf0 0x7f30c41f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40ceed0 con 0x7f30c40c4ac0 2024-09-21T02:10:29.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40ced50 con 0x7f30c406e2c0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f30c40cf050 con 0x7f30c40cdbf0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 0x7f30c40c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 0x7f30c40c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46970/0 (socket says 172.21.15.96:46970) 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 -- 172.21.15.96:0/1559493578 learned_addr learned my addr 172.21.15.96:0/1559493578 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c37fe640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30c406e2c0 0x7f30c415fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c3fff640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30c40cdbf0 0x7f30c41f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 -- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30c40cdbf0 msgr2=0x7f30c41f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30c40cdbf0 0x7f30c41f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 -- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30c406e2c0 msgr2=0x7f30c415fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30c406e2c0 0x7f30c415fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 -- 172.21.15.96:0/1559493578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c40c9ac0 con 0x7f30c40c4ac0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c37fe640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30c406e2c0 0x7f30c415fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30baffd640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 0x7f30c40c9580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b410 tx=0x7f30ac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:29.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac015020 con 0x7f30c40c4ac0 2024-09-21T02:10:29.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30ac011920 con 0x7f30c40c4ac0 2024-09-21T02:10:29.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c41f7950 con 0x7f30c40c4ac0 2024-09-21T02:10:29.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac011d70 con 0x7f30c40c4ac0 2024-09-21T02:10:29.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c40c8d80 con 0x7f30c40c4ac0 2024-09-21T02:10:29.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c40c4ed0 con 0x7f30c40c4ac0 2024-09-21T02:10:29.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30ac018020 con 0x7f30c40c4ac0 2024-09-21T02:10:29.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c0ff9640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f30a4073230 0x7f30a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:29.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f30ac0941d0 con 0x7f30c40c4ac0 2024-09-21T02:10:29.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.919+0000 7f30c37fe640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f30a4073230 0x7f30a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:29.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.923+0000 7f30c37fe640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f30a4073230 0x7f30a4075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30b40af000 tx=0x7f30b401b090 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:29.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:29.923+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f30ac01f020 con 0x7f30c40c4ac0 2024-09-21T02:10:30.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.099+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30c40b19d0 con 0x7f30c40c4ac0 2024-09-21T02:10:30.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.099+0000 7f30c0ff9640 1 -- 172.21.15.96:0/1559493578 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17002 (secure 0 0 0) 0x7f30ac05f060 con 0x7f30c40c4ac0 2024-09-21T02:10:30.107 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:30.107 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":40,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:10:27.276930+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:10:23.405387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T02:10:24.661586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f30a4073230 msgr2=0x7f30a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f30a4073230 0x7f30a4075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30b40af000 tx=0x7f30b401b090 comp rx=0 tx=0).stop 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 msgr2=0x7f30c40c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 0x7f30c40c9580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b410 tx=0x7f30ac009f90 comp rx=0 tx=0).stop 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 shutdown_connections 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f30a4073230 0x7f30a4075740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30c40cdbf0 0x7f30c41f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30c40c4ac0 0x7f30c40c9580 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 --2- 172.21.15.96:0/1559493578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30c406e2c0 0x7f30c415fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 >> 172.21.15.96:0/1559493578 conn(0x7f30c406eea0 msgr2=0x7f30c415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:30.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 shutdown_connections 2024-09-21T02:10:30.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.103+0000 7f30ca1c7640 1 -- 172.21.15.96:0/1559493578 wait complete. 2024-09-21T02:10:30.128 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:10:30.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 -- 172.21.15.96:0/3791983592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c073970 msgr2=0x7f249c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 --2- 172.21.15.96:0/3791983592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c073970 0x7f249c06dea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f249c0afaf0 tx=0x7f249001c670 comp rx=0 tx=0).stop 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 -- 172.21.15.96:0/3791983592 shutdown_connections 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 --2- 172.21.15.96:0/3791983592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c073970 0x7f249c06dea0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 --2- 172.21.15.96:0/3791983592 >> v2:172.21.15.122:3300/0 conn(0x7f249c155240 0x7f249c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 -- 172.21.15.96:0/3791983592 >> 172.21.15.96:0/3791983592 conn(0x7f249c06eea0 msgr2=0x7f249c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 -- 172.21.15.96:0/3791983592 shutdown_connections 2024-09-21T02:10:30.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 -- 172.21.15.96:0/3791983592 wait complete. 2024-09-21T02:10:30.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 Processor -- start 2024-09-21T02:10:30.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.263+0000 7f24a19d5640 1 -- start start 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249c06e3e0 0x7f249c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249c073970 0x7f249c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 0x7f249c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f249c165ef0 con 0x7f249c155240 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f249c165d70 con 0x7f249c073970 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f249c166070 con 0x7f249c06e3e0 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 0x7f249c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 0x7f249c15e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46984/0 (socket says 172.21.15.96:46984) 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 -- 172.21.15.96:0/1078470941 learned_addr learned my addr 172.21.15.96:0/1078470941 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249a7fc640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249c073970 0x7f249c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249affd640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249c06e3e0 0x7f249c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 -- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249c06e3e0 msgr2=0x7f249c0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249c06e3e0 0x7f249c0c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 -- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249c073970 msgr2=0x7f249c150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249c073970 0x7f249c150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 -- 172.21.15.96:0/1078470941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c0b3f40 con 0x7f249c155240 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249affd640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249c06e3e0 0x7f249c0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f249b7fe640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 0x7f249c15e0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f249400b970 tx=0x7f249400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494019070 con 0x7f249c155240 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f249400cc30 con 0x7f249c155240 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249c0c77a0 con 0x7f249c155240 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940146a0 con 0x7f249c155240 2024-09-21T02:10:30.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249c1fab40 con 0x7f249c155240 2024-09-21T02:10:30.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249c06dea0 con 0x7f249c155240 2024-09-21T02:10:30.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.267+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f249401c070 con 0x7f249c155240 2024-09-21T02:10:30.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.271+0000 7f2483fff640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2488073200 0x7f2488075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.271+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2494094140 con 0x7f249c155240 2024-09-21T02:10:30.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.271+0000 7f249affd640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2488073200 0x7f2488075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.271+0000 7f249affd640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2488073200 0x7f2488075710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f248c005fd0 tx=0x7f248c005dc0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:30.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.275+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f249405f060 con 0x7f249c155240 2024-09-21T02:10:30.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.443+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f249c06d670 con 0x7f2488073200 2024-09-21T02:10:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.455+0000 7f2483fff640 1 -- 172.21.15.96:0/1078470941 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240062 (secure 0 0 0) 0x7f249c06d670 con 0x7f2488073200 2024-09-21T02:10:30.463 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2488073200 msgr2=0x7f2488075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2488073200 0x7f2488075710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f248c005fd0 tx=0x7f248c005dc0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 msgr2=0x7f249c15e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 0x7f249c15e0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f249400b970 tx=0x7f249400be30 comp rx=0 tx=0).stop 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 shutdown_connections 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2488073200 0x7f2488075710 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f249c155240 0x7f249c15e0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249c073970 0x7f249c150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 --2- 172.21.15.96:0/1078470941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249c06e3e0 0x7f249c0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 >> 172.21.15.96:0/1078470941 conn(0x7f249c06eea0 msgr2=0x7f249c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 shutdown_connections 2024-09-21T02:10:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.459+0000 7f24a19d5640 1 -- 172.21.15.96:0/1078470941 wait complete. 2024-09-21T02:10:30.483 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-21T02:10:28.528243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228952,"kb_used_data":4372,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745768,"statfs":{"total":773094113280,"available":772859666432,"internally_reserved":0,"allocated":4476928,"data_stored":2555961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.264485"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.274986+0000","last_change":"2024-09-21T02:10:24.274986+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.274986+0000","last_undegraded":"2024-09-21T02:10:24.274986+0000","last_fullsized":"2024-09-21T02:10:24.274986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.274540+0000","last_change":"2024-09-21T02:10:24.274540+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.274540+0000","last_undegraded":"2024-09-21T02:10:24.274540+0000","last_fullsized":"2024-09-21T02:10:24.274540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.279650+0000","last_change":"2024-09-21T02:10:24.279650+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.279650+0000","last_undegraded":"2024-09-21T02:10:24.279650+0000","last_fullsized":"2024-09-21T02:10:24.279650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.268453+0000","last_change":"2024-09-21T02:10:24.268453+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.268453+0000","last_undegraded":"2024-09-21T02:10:24.268453+0000","last_fullsized":"2024-09-21T02:10:24.268453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.277170+0000","last_change":"2024-09-21T02:10:24.277170+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.277170+0000","last_undegraded":"2024-09-21T02:10:24.277170+0000","last_fullsized":"2024-09-21T02:10:24.277170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.269514+0000","last_change":"2024-09-21T02:10:24.269514+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.269514+0000","last_undegraded":"2024-09-21T02:10:24.269514+0000","last_fullsized":"2024-09-21T02:10:24.269514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.279066+0000","last_change":"2024-09-21T02:10:24.279066+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.279066+0000","last_undegraded":"2024-09-21T02:10:24.279066+0000","last_fullsized":"2024-09-21T02:10:24.279066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.268390+0000","last_change":"2024-09-21T02:10:24.268390+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.268390+0000","last_undegraded":"2024-09-21T02:10:24.268390+0000","last_fullsized":"2024-09-21T02:10:24.268390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.274513+0000","last_change":"2024-09-21T02:10:24.274513+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.274513+0000","last_undegraded":"2024-09-21T02:10:24.274513+0000","last_fullsized":"2024-09-21T02:10:24.274513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.277799+0000","last_change":"2024-09-21T02:10:24.277799+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.277799+0000","last_undegraded":"2024-09-21T02:10:24.277799+0000","last_fullsized":"2024-09-21T02:10:24.277799+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.277010+0000","last_change":"2024-09-21T02:10:24.277010+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.277010+0000","last_undegraded":"2024-09-21T02:10:24.277010+0000","last_fullsized":"2024-09-21T02:10:24.277010+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.273120+0000","last_change":"2024-09-21T02:10:24.273120+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.273120+0000","last_undegraded":"2024-09-21T02:10:24.273120+0000","last_fullsized":"2024-09-21T02:10:24.273120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.273314+0000","last_change":"2024-09-21T02:10:24.273314+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.273314+0000","last_undegraded":"2024-09-21T02:10:24.273314+0000","last_fullsized":"2024-09-21T02:10:24.273314+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.278260+0000","last_change":"2024-09-21T02:10:24.278260+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.278260+0000","last_undegraded":"2024-09-21T02:10:24.278260+0000","last_fullsized":"2024-09-21T02:10:24.278260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.274380+0000","last_change":"2024-09-21T02:10:24.274380+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.274380+0000","last_undegraded":"2024-09-21T02:10:24.274380+0000","last_fullsized":"2024-09-21T02:10:24.274380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.267865+0000","last_change":"2024-09-21T02:10:24.267865+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.267865+0000","last_undegraded":"2024-09-21T02:10:24.267865+0000","last_fullsized":"2024-09-21T02:10:24.267865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.278984+0000","last_change":"2024-09-21T02:10:24.278984+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.278984+0000","last_undegraded":"2024-09-21T02:10:24.278984+0000","last_fullsized":"2024-09-21T02:10:24.278984+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.269394+0000","last_change":"2024-09-21T02:10:24.269394+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.269394+0000","last_undegraded":"2024-09-21T02:10:24.269394+0000","last_fullsized":"2024-09-21T02:10:24.269394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.269284+0000","last_change":"2024-09-21T02:10:24.269284+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.269284+0000","last_undegraded":"2024-09-21T02:10:24.269284+0000","last_fullsized":"2024-09-21T02:10:24.269284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.278731+0000","last_change":"2024-09-21T02:10:24.278731+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.278731+0000","last_undegraded":"2024-09-21T02:10:24.278731+0000","last_fullsized":"2024-09-21T02:10:24.278731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.269357+0000","last_change":"2024-09-21T02:10:24.269357+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.269357+0000","last_undegraded":"2024-09-21T02:10:24.269357+0000","last_fullsized":"2024-09-21T02:10:24.269357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.269045+0000","last_change":"2024-09-21T02:10:24.269045+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.269045+0000","last_undegraded":"2024-09-21T02:10:24.269045+0000","last_fullsized":"2024-09-21T02:10:24.269045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.268037+0000","last_change":"2024-09-21T02:10:24.268037+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.268037+0000","last_undegraded":"2024-09-21T02:10:24.268037+0000","last_fullsized":"2024-09-21T02:10:24.268037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.269080+0000","last_change":"2024-09-21T02:10:24.269080+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.269080+0000","last_undegraded":"2024-09-21T02:10:24.269080+0000","last_fullsized":"2024-09-21T02:10:24.269080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.277125+0000","last_change":"2024-09-21T02:10:24.277125+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.277125+0000","last_undegraded":"2024-09-21T02:10:24.277125+0000","last_fullsized":"2024-09-21T02:10:24.277125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.268955+0000","last_change":"2024-09-21T02:10:24.268955+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.268955+0000","last_undegraded":"2024-09-21T02:10:24.268955+0000","last_fullsized":"2024-09-21T02:10:24.268955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.276062+0000","last_change":"2024-09-21T02:10:24.276062+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.276062+0000","last_undegraded":"2024-09-21T02:10:24.276062+0000","last_fullsized":"2024-09-21T02:10:24.276062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.275022+0000","last_change":"2024-09-21T02:10:24.275022+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.275022+0000","last_undegraded":"2024-09-21T02:10:24.275022+0000","last_fullsized":"2024-09-21T02:10:24.275022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.274455+0000","last_change":"2024-09-21T02:10:24.274455+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.274455+0000","last_undegraded":"2024-09-21T02:10:24.274455+0000","last_fullsized":"2024-09-21T02:10:24.274455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.270657+0000","last_change":"2024-09-21T02:10:24.270657+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.270657+0000","last_undegraded":"2024-09-21T02:10:24.270657+0000","last_fullsized":"2024-09-21T02:10:24.270657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.272921+0000","last_change":"2024-09-21T02:10:24.272921+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.272921+0000","last_undegraded":"2024-09-21T02:10:24.272921+0000","last_fullsized":"2024-09-21T02:10:24.272921+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:10:25.260288+0000","last_change":"2024-09-21T02:10:25.260288+0000","last_active":"2024-09-21T02:10:25.260288+0000","last_peered":"2024-09-21T02:10:25.260288+0000","last_clean":"2024-09-21T02:10:25.260288+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:25.260288+0000","last_undegraded":"2024-09-21T02:10:25.260288+0000","last_fullsized":"2024-09-21T02:10:25.260288+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-21T02:10:24.270965+0000","last_change":"2024-09-21T02:10:24.270965+0000","last_active":"2024-09-21T02:10:24.256097+0000","last_peered":"2024-09-21T02:10:24.256097+0000","last_clean":"2024-09-21T02:10:24.256097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:10:24.270965+0000","last_undegraded":"2024-09-21T02:10:24.270965+0000","last_fullsized":"2024-09-21T02:10:24.270965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":141,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T02:10:24.269191+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:10:24.269191+0000","last_peered":"2024-09-21T02:10:24.269191+0000","last_clean":"2024-09-21T02:10:24.269191+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:10:24.269191+0000","last_undegraded":"2024-09-21T02:10:24.269191+0000","last_fullsized":"2024-09-21T02:10:24.269191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1015808,"data_stored":767704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":171127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":409600,"data_stored":171127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":409600,"data_stored":171127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":409600,"data_stored":171127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":409600,"data_stored":171127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":409600,"data_stored":171127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":761495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:10:30.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:10:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb664603640 1 -- 172.21.15.96:0/87665904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb648004810 msgr2=0x7fb648004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb664603640 1 --2- 172.21.15.96:0/87665904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb648004810 0x7fb648004c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb658009b60 tx=0x7fb65801d240 comp rx=0 tx=0).stop 2024-09-21T02:10:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb662ba0640 1 -- 172.21.15.96:0/87665904 reap_dead start 2024-09-21T02:10:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb664603640 1 -- 172.21.15.96:0/87665904 shutdown_connections 2024-09-21T02:10:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb664603640 1 -- 172.21.15.96:0/87665904 >> 172.21.15.96:0/87665904 conn(0x7fb65c0cf110 msgr2=0x7fb65c0cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb664603640 1 -- 172.21.15.96:0/87665904 shutdown_connections 2024-09-21T02:10:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.671+0000 7fb664603640 1 -- 172.21.15.96:0/87665904 wait complete. 2024-09-21T02:10:30.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 Processor -- start 2024-09-21T02:10:30.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- start start 2024-09-21T02:10:30.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb648004810 0x7fb65c0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 0x7fb65c1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb65c1f6d70 0x7fb65c1f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c0b2a40 con 0x7fb65c0c87c0 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c0b28c0 con 0x7fb648004810 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb65c152e50 con 0x7fb65c1f6d70 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 0x7fb65c1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 0x7fb65c1f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46992/0 (socket says 172.21.15.96:46992) 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 -- 172.21.15.96:0/628605007 learned_addr learned my addr 172.21.15.96:0/628605007 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb66239f640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb648004810 0x7fb65c0c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 -- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb65c1f6d70 msgr2=0x7fb65c1f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb662ba0640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb65c1f6d70 0x7fb65c1f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb65c1f6d70 0x7fb65c1f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 -- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb648004810 msgr2=0x7fb65c0c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb648004810 0x7fb65c0c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 -- 172.21.15.96:0/628605007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c0cbf00 con 0x7fb65c0c87c0 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb662ba0640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb65c1f6d70 0x7fb65c1f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb661b9e640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 0x7fb65c1f6830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb658009a80 tx=0x7fb658005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580ba020 con 0x7fb65c0c87c0 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6580ac8a0 con 0x7fb65c0c87c0 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c0c8bd0 con 0x7fb65c0c87c0 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580b1040 con 0x7fb65c0c87c0 2024-09-21T02:10:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c0ca240 con 0x7fb65c0c87c0 2024-09-21T02:10:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.675+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c0d0570 con 0x7fb65c0c87c0 2024-09-21T02:10:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.679+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb6580bd020 con 0x7fb65c0c87c0 2024-09-21T02:10:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.679+0000 7fb652ffd640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb640073230 0x7fb640075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.679+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fb65812f790 con 0x7fb65c0c87c0 2024-09-21T02:10:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.679+0000 7fb66239f640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb640073230 0x7fb640075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.679+0000 7fb66239f640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb640073230 0x7fb640075740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005ec0 tx=0x7fb64c005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.683+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb6580c4030 con 0x7fb65c0c87c0 2024-09-21T02:10:30.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.839+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb65c0c66d0 con 0x7fb640073230 2024-09-21T02:10:30.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.851+0000 7fb652ffd640 1 -- 172.21.15.96:0/628605007 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251956 (secure 0 0 0) 0x7fb65c0c66d0 con 0x7fb640073230 2024-09-21T02:10:30.858 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:30.859 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb640073230 msgr2=0x7fb640075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb640073230 0x7fb640075740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005ec0 tx=0x7fb64c005860 comp rx=0 tx=0).stop 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 msgr2=0x7fb65c1f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 0x7fb65c1f6830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb658009a80 tx=0x7fb658005640 comp rx=0 tx=0).stop 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 shutdown_connections 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb640073230 0x7fb640075740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb65c1f6d70 0x7fb65c1f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c0c87c0 0x7fb65c1f6830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 --2- 172.21.15.96:0/628605007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb648004810 0x7fb65c0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 >> 172.21.15.96:0/628605007 conn(0x7fb65c0cf110 msgr2=0x7fb65c071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 shutdown_connections 2024-09-21T02:10:30.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:30.855+0000 7fb664603640 1 -- 172.21.15.96:0/628605007 wait complete. 2024-09-21T02:10:30.877 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-21T02:10:30.528900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229384,"kb_used_data":4808,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745336,"statfs":{"total":773094113280,"available":772859224064,"internally_reserved":0,"allocated":4923392,"data_stored":2768452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001868"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.287338+0000","last_change":"2024-09-21T02:10:26.308137+0000","last_active":"2024-09-21T02:10:27.287338+0000","last_peered":"2024-09-21T02:10:27.287338+0000","last_clean":"2024-09-21T02:10:27.287338+0000","last_became_active":"2024-09-21T02:10:26.307826+0000","last_became_peered":"2024-09-21T02:10:26.307826+0000","last_unstale":"2024-09-21T02:10:27.287338+0000","last_undegraded":"2024-09-21T02:10:27.287338+0000","last_fullsized":"2024-09-21T02:10:27.287338+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:50.487552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286924+0000","last_change":"2024-09-21T02:10:26.311252+0000","last_active":"2024-09-21T02:10:27.286924+0000","last_peered":"2024-09-21T02:10:27.286924+0000","last_clean":"2024-09-21T02:10:27.286924+0000","last_became_active":"2024-09-21T02:10:26.310942+0000","last_became_peered":"2024-09-21T02:10:26.310942+0000","last_unstale":"2024-09-21T02:10:27.286924+0000","last_undegraded":"2024-09-21T02:10:27.286924+0000","last_fullsized":"2024-09-21T02:10:27.286924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.694455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.873077+0000","last_change":"2024-09-21T02:10:26.333016+0000","last_active":"2024-09-21T02:10:27.873077+0000","last_peered":"2024-09-21T02:10:27.873077+0000","last_clean":"2024-09-21T02:10:27.873077+0000","last_became_active":"2024-09-21T02:10:26.332746+0000","last_became_peered":"2024-09-21T02:10:26.332746+0000","last_unstale":"2024-09-21T02:10:27.873077+0000","last_undegraded":"2024-09-21T02:10:27.873077+0000","last_fullsized":"2024-09-21T02:10:27.873077+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:07.011062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.287383+0000","last_change":"2024-09-21T02:10:26.334678+0000","last_active":"2024-09-21T02:10:28.287383+0000","last_peered":"2024-09-21T02:10:28.287383+0000","last_clean":"2024-09-21T02:10:28.287383+0000","last_became_active":"2024-09-21T02:10:26.334314+0000","last_became_peered":"2024-09-21T02:10:26.334314+0000","last_unstale":"2024-09-21T02:10:28.287383+0000","last_undegraded":"2024-09-21T02:10:28.287383+0000","last_fullsized":"2024-09-21T02:10:28.287383+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:17.598074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.286946+0000","last_change":"2024-09-21T02:10:26.334718+0000","last_active":"2024-09-21T02:10:28.286946+0000","last_peered":"2024-09-21T02:10:28.286946+0000","last_clean":"2024-09-21T02:10:28.286946+0000","last_became_active":"2024-09-21T02:10:26.333376+0000","last_became_peered":"2024-09-21T02:10:26.333376+0000","last_unstale":"2024-09-21T02:10:28.286946+0000","last_undegraded":"2024-09-21T02:10:28.286946+0000","last_fullsized":"2024-09-21T02:10:28.286946+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.365183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599400+0000","last_change":"2024-09-21T02:10:26.297350+0000","last_active":"2024-09-21T02:10:27.599400+0000","last_peered":"2024-09-21T02:10:27.599400+0000","last_clean":"2024-09-21T02:10:27.599400+0000","last_became_active":"2024-09-21T02:10:26.297021+0000","last_became_peered":"2024-09-21T02:10:26.297021+0000","last_unstale":"2024-09-21T02:10:27.599400+0000","last_undegraded":"2024-09-21T02:10:27.599400+0000","last_fullsized":"2024-09-21T02:10:27.599400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:57.932758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.293763+0000","last_change":"2024-09-21T02:10:26.344753+0000","last_active":"2024-09-21T02:10:28.293763+0000","last_peered":"2024-09-21T02:10:28.293763+0000","last_clean":"2024-09-21T02:10:28.293763+0000","last_became_active":"2024-09-21T02:10:26.343936+0000","last_became_peered":"2024-09-21T02:10:26.343936+0000","last_unstale":"2024-09-21T02:10:28.293763+0000","last_undegraded":"2024-09-21T02:10:28.293763+0000","last_fullsized":"2024-09-21T02:10:28.293763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:07.251515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.285775+0000","last_change":"2024-09-21T02:10:26.333807+0000","last_active":"2024-09-21T02:10:27.285775+0000","last_peered":"2024-09-21T02:10:27.285775+0000","last_clean":"2024-09-21T02:10:27.285775+0000","last_became_active":"2024-09-21T02:10:26.333470+0000","last_became_peered":"2024-09-21T02:10:26.333470+0000","last_unstale":"2024-09-21T02:10:27.285775+0000","last_undegraded":"2024-09-21T02:10:27.285775+0000","last_fullsized":"2024-09-21T02:10:27.285775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.784063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.873094+0000","last_change":"2024-09-21T02:10:26.351067+0000","last_active":"2024-09-21T02:10:27.873094+0000","last_peered":"2024-09-21T02:10:27.873094+0000","last_clean":"2024-09-21T02:10:27.873094+0000","last_became_active":"2024-09-21T02:10:26.350777+0000","last_became_peered":"2024-09-21T02:10:26.350777+0000","last_unstale":"2024-09-21T02:10:27.873094+0000","last_undegraded":"2024-09-21T02:10:27.873094+0000","last_fullsized":"2024-09-21T02:10:27.873094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:23.668030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294553+0000","last_change":"2024-09-21T02:10:26.320875+0000","last_active":"2024-09-21T02:10:28.294553+0000","last_peered":"2024-09-21T02:10:28.294553+0000","last_clean":"2024-09-21T02:10:28.294553+0000","last_became_active":"2024-09-21T02:10:26.320094+0000","last_became_peered":"2024-09-21T02:10:26.320094+0000","last_unstale":"2024-09-21T02:10:28.294553+0000","last_undegraded":"2024-09-21T02:10:28.294553+0000","last_fullsized":"2024-09-21T02:10:28.294553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.784063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294993+0000","last_change":"2024-09-21T02:10:26.344390+0000","last_active":"2024-09-21T02:10:28.294993+0000","last_peered":"2024-09-21T02:10:28.294993+0000","last_clean":"2024-09-21T02:10:28.294993+0000","last_became_active":"2024-09-21T02:10:26.343564+0000","last_became_peered":"2024-09-21T02:10:26.343564+0000","last_unstale":"2024-09-21T02:10:28.294993+0000","last_undegraded":"2024-09-21T02:10:28.294993+0000","last_fullsized":"2024-09-21T02:10:28.294993+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:52.593492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872741+0000","last_change":"2024-09-21T02:10:26.311111+0000","last_active":"2024-09-21T02:10:27.872741+0000","last_peered":"2024-09-21T02:10:27.872741+0000","last_clean":"2024-09-21T02:10:27.872741+0000","last_became_active":"2024-09-21T02:10:26.310645+0000","last_became_peered":"2024-09-21T02:10:26.310645+0000","last_unstale":"2024-09-21T02:10:27.872741+0000","last_undegraded":"2024-09-21T02:10:27.872741+0000","last_fullsized":"2024-09-21T02:10:27.872741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:24.178726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294297+0000","last_change":"2024-09-21T02:10:26.344852+0000","last_active":"2024-09-21T02:10:28.294297+0000","last_peered":"2024-09-21T02:10:28.294297+0000","last_clean":"2024-09-21T02:10:28.294297+0000","last_became_active":"2024-09-21T02:10:26.307589+0000","last_became_peered":"2024-09-21T02:10:26.307589+0000","last_unstale":"2024-09-21T02:10:28.294297+0000","last_undegraded":"2024-09-21T02:10:28.294297+0000","last_fullsized":"2024-09-21T02:10:28.294297+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:27.394758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.613494+0000","last_change":"2024-09-21T02:10:26.354889+0000","last_active":"2024-09-21T02:10:27.613494+0000","last_peered":"2024-09-21T02:10:27.613494+0000","last_clean":"2024-09-21T02:10:27.613494+0000","last_became_active":"2024-09-21T02:10:26.354349+0000","last_became_peered":"2024-09-21T02:10:26.354349+0000","last_unstale":"2024-09-21T02:10:27.613494+0000","last_undegraded":"2024-09-21T02:10:27.613494+0000","last_fullsized":"2024-09-21T02:10:27.613494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:34.190730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294849+0000","last_change":"2024-09-21T02:10:26.333968+0000","last_active":"2024-09-21T02:10:28.294849+0000","last_peered":"2024-09-21T02:10:28.294849+0000","last_clean":"2024-09-21T02:10:28.294849+0000","last_became_active":"2024-09-21T02:10:26.333268+0000","last_became_peered":"2024-09-21T02:10:26.333268+0000","last_unstale":"2024-09-21T02:10:28.294849+0000","last_undegraded":"2024-09-21T02:10:28.294849+0000","last_fullsized":"2024-09-21T02:10:28.294849+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.297071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.288741+0000","last_change":"2024-09-21T02:10:26.330331+0000","last_active":"2024-09-21T02:10:27.288741+0000","last_peered":"2024-09-21T02:10:27.288741+0000","last_clean":"2024-09-21T02:10:27.288741+0000","last_became_active":"2024-09-21T02:10:26.327802+0000","last_became_peered":"2024-09-21T02:10:26.327802+0000","last_unstale":"2024-09-21T02:10:27.288741+0000","last_undegraded":"2024-09-21T02:10:27.288741+0000","last_fullsized":"2024-09-21T02:10:27.288741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:04.056161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290778+0000","last_change":"2024-09-21T02:10:26.336440+0000","last_active":"2024-09-21T02:10:27.290778+0000","last_peered":"2024-09-21T02:10:27.290778+0000","last_clean":"2024-09-21T02:10:27.290778+0000","last_became_active":"2024-09-21T02:10:26.333000+0000","last_became_peered":"2024-09-21T02:10:26.333000+0000","last_unstale":"2024-09-21T02:10:27.290778+0000","last_undegraded":"2024-09-21T02:10:27.290778+0000","last_fullsized":"2024-09-21T02:10:27.290778+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:25.676971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290762+0000","last_change":"2024-09-21T02:10:26.353838+0000","last_active":"2024-09-21T02:10:27.290762+0000","last_peered":"2024-09-21T02:10:27.290762+0000","last_clean":"2024-09-21T02:10:27.290762+0000","last_became_active":"2024-09-21T02:10:26.353224+0000","last_became_peered":"2024-09-21T02:10:26.353224+0000","last_unstale":"2024-09-21T02:10:27.290762+0000","last_undegraded":"2024-09-21T02:10:27.290762+0000","last_fullsized":"2024-09-21T02:10:27.290762+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:06.655175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.287322+0000","last_change":"2024-09-21T02:10:26.308258+0000","last_active":"2024-09-21T02:10:28.287322+0000","last_peered":"2024-09-21T02:10:28.287322+0000","last_clean":"2024-09-21T02:10:28.287322+0000","last_became_active":"2024-09-21T02:10:26.307841+0000","last_became_peered":"2024-09-21T02:10:26.307841+0000","last_unstale":"2024-09-21T02:10:28.287322+0000","last_undegraded":"2024-09-21T02:10:28.287322+0000","last_fullsized":"2024-09-21T02:10:28.287322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:22.831668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.289644+0000","last_change":"2024-09-21T02:10:26.354895+0000","last_active":"2024-09-21T02:10:27.289644+0000","last_peered":"2024-09-21T02:10:27.289644+0000","last_clean":"2024-09-21T02:10:27.289644+0000","last_became_active":"2024-09-21T02:10:26.353843+0000","last_became_peered":"2024-09-21T02:10:26.353843+0000","last_unstale":"2024-09-21T02:10:27.289644+0000","last_undegraded":"2024-09-21T02:10:27.289644+0000","last_fullsized":"2024-09-21T02:10:27.289644+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:20.855759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.295109+0000","last_change":"2024-09-21T02:10:26.335028+0000","last_active":"2024-09-21T02:10:28.295109+0000","last_peered":"2024-09-21T02:10:28.295109+0000","last_clean":"2024-09-21T02:10:28.295109+0000","last_became_active":"2024-09-21T02:10:26.318347+0000","last_became_peered":"2024-09-21T02:10:26.318347+0000","last_unstale":"2024-09-21T02:10:28.295109+0000","last_undegraded":"2024-09-21T02:10:28.295109+0000","last_fullsized":"2024-09-21T02:10:28.295109+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:06.917597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.614516+0000","last_change":"2024-09-21T02:10:26.333641+0000","last_active":"2024-09-21T02:10:27.614516+0000","last_peered":"2024-09-21T02:10:27.614516+0000","last_clean":"2024-09-21T02:10:27.614516+0000","last_became_active":"2024-09-21T02:10:26.332770+0000","last_became_peered":"2024-09-21T02:10:26.332770+0000","last_unstale":"2024-09-21T02:10:27.614516+0000","last_undegraded":"2024-09-21T02:10:27.614516+0000","last_fullsized":"2024-09-21T02:10:27.614516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:16.103922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290309+0000","last_change":"2024-09-21T02:10:26.311098+0000","last_active":"2024-09-21T02:10:27.290309+0000","last_peered":"2024-09-21T02:10:27.290309+0000","last_clean":"2024-09-21T02:10:27.290309+0000","last_became_active":"2024-09-21T02:10:26.310228+0000","last_became_peered":"2024-09-21T02:10:26.310228+0000","last_unstale":"2024-09-21T02:10:27.290309+0000","last_undegraded":"2024-09-21T02:10:27.290309+0000","last_fullsized":"2024-09-21T02:10:27.290309+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:38.362611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.286599+0000","last_change":"2024-09-21T02:10:26.303662+0000","last_active":"2024-09-21T02:10:28.286599+0000","last_peered":"2024-09-21T02:10:28.286599+0000","last_clean":"2024-09-21T02:10:28.286599+0000","last_became_active":"2024-09-21T02:10:26.302906+0000","last_became_peered":"2024-09-21T02:10:26.302906+0000","last_unstale":"2024-09-21T02:10:28.286599+0000","last_undegraded":"2024-09-21T02:10:28.286599+0000","last_fullsized":"2024-09-21T02:10:28.286599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:24:02.297619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.291874+0000","last_change":"2024-09-21T02:10:26.327049+0000","last_active":"2024-09-21T02:10:27.291874+0000","last_peered":"2024-09-21T02:10:27.291874+0000","last_clean":"2024-09-21T02:10:27.291874+0000","last_became_active":"2024-09-21T02:10:26.315366+0000","last_became_peered":"2024-09-21T02:10:26.315366+0000","last_unstale":"2024-09-21T02:10:27.291874+0000","last_undegraded":"2024-09-21T02:10:27.291874+0000","last_fullsized":"2024-09-21T02:10:27.291874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:26.249535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599864+0000","last_change":"2024-09-21T02:10:26.333910+0000","last_active":"2024-09-21T02:10:27.599864+0000","last_peered":"2024-09-21T02:10:27.599864+0000","last_clean":"2024-09-21T02:10:27.599864+0000","last_became_active":"2024-09-21T02:10:26.333532+0000","last_became_peered":"2024-09-21T02:10:26.333532+0000","last_unstale":"2024-09-21T02:10:27.599864+0000","last_undegraded":"2024-09-21T02:10:27.599864+0000","last_fullsized":"2024-09-21T02:10:27.599864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:28.177088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290538+0000","last_change":"2024-09-21T02:10:26.351761+0000","last_active":"2024-09-21T02:10:27.290538+0000","last_peered":"2024-09-21T02:10:27.290538+0000","last_clean":"2024-09-21T02:10:27.290538+0000","last_became_active":"2024-09-21T02:10:26.346131+0000","last_became_peered":"2024-09-21T02:10:26.346131+0000","last_unstale":"2024-09-21T02:10:27.290538+0000","last_undegraded":"2024-09-21T02:10:27.290538+0000","last_fullsized":"2024-09-21T02:10:27.290538+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:08.486302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.295192+0000","last_change":"2024-09-21T02:10:26.321811+0000","last_active":"2024-09-21T02:10:28.295192+0000","last_peered":"2024-09-21T02:10:28.295192+0000","last_clean":"2024-09-21T02:10:28.295192+0000","last_became_active":"2024-09-21T02:10:26.320478+0000","last_became_peered":"2024-09-21T02:10:26.320478+0000","last_unstale":"2024-09-21T02:10:28.295192+0000","last_undegraded":"2024-09-21T02:10:28.295192+0000","last_fullsized":"2024-09-21T02:10:28.295192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:03.900690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.291126+0000","last_change":"2024-09-21T02:10:26.325314+0000","last_active":"2024-09-21T02:10:27.291126+0000","last_peered":"2024-09-21T02:10:27.291126+0000","last_clean":"2024-09-21T02:10:27.291126+0000","last_became_active":"2024-09-21T02:10:26.324238+0000","last_became_peered":"2024-09-21T02:10:26.324238+0000","last_unstale":"2024-09-21T02:10:27.291126+0000","last_undegraded":"2024-09-21T02:10:27.291126+0000","last_fullsized":"2024-09-21T02:10:27.291126+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:36.255299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.289421+0000","last_change":"2024-09-21T02:10:26.327278+0000","last_active":"2024-09-21T02:10:27.289421+0000","last_peered":"2024-09-21T02:10:27.289421+0000","last_clean":"2024-09-21T02:10:27.289421+0000","last_became_active":"2024-09-21T02:10:26.326380+0000","last_became_peered":"2024-09-21T02:10:26.326380+0000","last_unstale":"2024-09-21T02:10:27.289421+0000","last_undegraded":"2024-09-21T02:10:27.289421+0000","last_fullsized":"2024-09-21T02:10:27.289421+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:59.082788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290912+0000","last_change":"2024-09-21T02:10:26.317606+0000","last_active":"2024-09-21T02:10:27.290912+0000","last_peered":"2024-09-21T02:10:27.290912+0000","last_clean":"2024-09-21T02:10:27.290912+0000","last_became_active":"2024-09-21T02:10:26.316735+0000","last_became_peered":"2024-09-21T02:10:26.316735+0000","last_unstale":"2024-09-21T02:10:27.290912+0000","last_undegraded":"2024-09-21T02:10:27.290912+0000","last_fullsized":"2024-09-21T02:10:27.290912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:05.724426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599347+0000","last_change":"2024-09-21T02:10:26.311371+0000","last_active":"2024-09-21T02:10:27.599347+0000","last_peered":"2024-09-21T02:10:27.599347+0000","last_clean":"2024-09-21T02:10:27.599347+0000","last_became_active":"2024-09-21T02:10:26.311056+0000","last_became_peered":"2024-09-21T02:10:26.311056+0000","last_unstale":"2024-09-21T02:10:27.599347+0000","last_undegraded":"2024-09-21T02:10:27.599347+0000","last_fullsized":"2024-09-21T02:10:27.599347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:58.130578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.291682+0000","last_change":"2024-09-21T02:10:26.336527+0000","last_active":"2024-09-21T02:10:27.291682+0000","last_peered":"2024-09-21T02:10:27.291682+0000","last_clean":"2024-09-21T02:10:27.291682+0000","last_became_active":"2024-09-21T02:10:26.335729+0000","last_became_peered":"2024-09-21T02:10:26.335729+0000","last_unstale":"2024-09-21T02:10:27.291682+0000","last_undegraded":"2024-09-21T02:10:27.291682+0000","last_fullsized":"2024-09-21T02:10:27.291682+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:42.991317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.292067+0000","last_change":"2024-09-21T02:10:25.329940+0000","last_active":"2024-09-21T02:10:27.292067+0000","last_peered":"2024-09-21T02:10:27.292067+0000","last_clean":"2024-09-21T02:10:27.292067+0000","last_became_active":"2024-09-21T02:10:25.329321+0000","last_became_peered":"2024-09-21T02:10:25.329321+0000","last_unstale":"2024-09-21T02:10:27.292067+0000","last_undegraded":"2024-09-21T02:10:27.292067+0000","last_fullsized":"2024-09-21T02:10:27.292067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:12.044100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599948+0000","last_change":"2024-09-21T02:10:26.348952+0000","last_active":"2024-09-21T02:10:27.599948+0000","last_peered":"2024-09-21T02:10:27.599948+0000","last_clean":"2024-09-21T02:10:27.599948+0000","last_became_active":"2024-09-21T02:10:26.348623+0000","last_became_peered":"2024-09-21T02:10:26.348623+0000","last_unstale":"2024-09-21T02:10:27.599948+0000","last_undegraded":"2024-09-21T02:10:27.599948+0000","last_fullsized":"2024-09-21T02:10:27.599948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:22:35.617470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872453+0000","last_change":"2024-09-21T02:10:25.315589+0000","last_active":"2024-09-21T02:10:27.872453+0000","last_peered":"2024-09-21T02:10:27.872453+0000","last_clean":"2024-09-21T02:10:27.872453+0000","last_became_active":"2024-09-21T02:10:25.315124+0000","last_became_peered":"2024-09-21T02:10:25.315124+0000","last_unstale":"2024-09-21T02:10:27.872453+0000","last_undegraded":"2024-09-21T02:10:27.872453+0000","last_fullsized":"2024-09-21T02:10:27.872453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:15.983563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.614284+0000","last_change":"2024-09-21T02:10:26.357877+0000","last_active":"2024-09-21T02:10:27.614284+0000","last_peered":"2024-09-21T02:10:27.614284+0000","last_clean":"2024-09-21T02:10:27.614284+0000","last_became_active":"2024-09-21T02:10:26.357352+0000","last_became_peered":"2024-09-21T02:10:26.357352+0000","last_unstale":"2024-09-21T02:10:27.614284+0000","last_undegraded":"2024-09-21T02:10:27.614284+0000","last_fullsized":"2024-09-21T02:10:27.614284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:10.175432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599964+0000","last_change":"2024-09-21T02:10:25.316210+0000","last_active":"2024-09-21T02:10:27.599964+0000","last_peered":"2024-09-21T02:10:27.599964+0000","last_clean":"2024-09-21T02:10:27.599964+0000","last_became_active":"2024-09-21T02:10:25.315751+0000","last_became_peered":"2024-09-21T02:10:25.315751+0000","last_unstale":"2024-09-21T02:10:27.599964+0000","last_undegraded":"2024-09-21T02:10:27.599964+0000","last_fullsized":"2024-09-21T02:10:27.599964+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:49.895270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286825+0000","last_change":"2024-09-21T02:10:26.337215+0000","last_active":"2024-09-21T02:10:27.286825+0000","last_peered":"2024-09-21T02:10:27.286825+0000","last_clean":"2024-09-21T02:10:27.286825+0000","last_became_active":"2024-09-21T02:10:26.336791+0000","last_became_peered":"2024-09-21T02:10:26.336791+0000","last_unstale":"2024-09-21T02:10:27.286825+0000","last_undegraded":"2024-09-21T02:10:27.286825+0000","last_fullsized":"2024-09-21T02:10:27.286825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:41.035277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.613650+0000","last_change":"2024-09-21T02:10:25.310788+0000","last_active":"2024-09-21T02:10:27.613650+0000","last_peered":"2024-09-21T02:10:27.613650+0000","last_clean":"2024-09-21T02:10:27.613650+0000","last_became_active":"2024-09-21T02:10:25.310118+0000","last_became_peered":"2024-09-21T02:10:25.310118+0000","last_unstale":"2024-09-21T02:10:27.613650+0000","last_undegraded":"2024-09-21T02:10:27.613650+0000","last_fullsized":"2024-09-21T02:10:27.613650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:06.053799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872175+0000","last_change":"2024-09-21T02:10:26.303999+0000","last_active":"2024-09-21T02:10:27.872175+0000","last_peered":"2024-09-21T02:10:27.872175+0000","last_clean":"2024-09-21T02:10:27.872175+0000","last_became_active":"2024-09-21T02:10:26.303668+0000","last_became_peered":"2024-09-21T02:10:26.303668+0000","last_unstale":"2024-09-21T02:10:27.872175+0000","last_undegraded":"2024-09-21T02:10:27.872175+0000","last_fullsized":"2024-09-21T02:10:27.872175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:04.239444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294503+0000","last_change":"2024-09-21T02:10:25.311069+0000","last_active":"2024-09-21T02:10:28.294503+0000","last_peered":"2024-09-21T02:10:28.294503+0000","last_clean":"2024-09-21T02:10:28.294503+0000","last_became_active":"2024-09-21T02:10:25.309513+0000","last_became_peered":"2024-09-21T02:10:25.309513+0000","last_unstale":"2024-09-21T02:10:28.294503+0000","last_undegraded":"2024-09-21T02:10:28.294503+0000","last_fullsized":"2024-09-21T02:10:28.294503+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:08.799194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294547+0000","last_change":"2024-09-21T02:10:26.319084+0000","last_active":"2024-09-21T02:10:28.294547+0000","last_peered":"2024-09-21T02:10:28.294547+0000","last_clean":"2024-09-21T02:10:28.294547+0000","last_became_active":"2024-09-21T02:10:26.308520+0000","last_became_peered":"2024-09-21T02:10:26.308520+0000","last_unstale":"2024-09-21T02:10:28.294547+0000","last_undegraded":"2024-09-21T02:10:28.294547+0000","last_fullsized":"2024-09-21T02:10:28.294547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:44.231387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290748+0000","last_change":"2024-09-21T02:10:25.331260+0000","last_active":"2024-09-21T02:10:27.290748+0000","last_peered":"2024-09-21T02:10:27.290748+0000","last_clean":"2024-09-21T02:10:27.290748+0000","last_became_active":"2024-09-21T02:10:25.330845+0000","last_became_peered":"2024-09-21T02:10:25.330845+0000","last_unstale":"2024-09-21T02:10:27.290748+0000","last_undegraded":"2024-09-21T02:10:27.290748+0000","last_fullsized":"2024-09-21T02:10:27.290748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:49.895270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.614064+0000","last_change":"2024-09-21T02:10:26.359343+0000","last_active":"2024-09-21T02:10:27.614064+0000","last_peered":"2024-09-21T02:10:27.614064+0000","last_clean":"2024-09-21T02:10:27.614064+0000","last_became_active":"2024-09-21T02:10:26.358758+0000","last_became_peered":"2024-09-21T02:10:26.358758+0000","last_unstale":"2024-09-21T02:10:27.614064+0000","last_undegraded":"2024-09-21T02:10:27.614064+0000","last_fullsized":"2024-09-21T02:10:27.614064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:36.070517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599003+0000","last_change":"2024-09-21T02:10:25.311555+0000","last_active":"2024-09-21T02:10:27.599003+0000","last_peered":"2024-09-21T02:10:27.599003+0000","last_clean":"2024-09-21T02:10:27.599003+0000","last_became_active":"2024-09-21T02:10:25.310863+0000","last_became_peered":"2024-09-21T02:10:25.310863+0000","last_unstale":"2024-09-21T02:10:27.599003+0000","last_undegraded":"2024-09-21T02:10:27.599003+0000","last_fullsized":"2024-09-21T02:10:27.599003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:12.763856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286093+0000","last_change":"2024-09-21T02:10:26.309266+0000","last_active":"2024-09-21T02:10:27.286093+0000","last_peered":"2024-09-21T02:10:27.286093+0000","last_clean":"2024-09-21T02:10:27.286093+0000","last_became_active":"2024-09-21T02:10:26.308941+0000","last_became_peered":"2024-09-21T02:10:26.308941+0000","last_unstale":"2024-09-21T02:10:27.286093+0000","last_undegraded":"2024-09-21T02:10:27.286093+0000","last_fullsized":"2024-09-21T02:10:27.286093+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:47.021655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.612860+0000","last_change":"2024-09-21T02:10:25.323653+0000","last_active":"2024-09-21T02:10:27.612860+0000","last_peered":"2024-09-21T02:10:27.612860+0000","last_clean":"2024-09-21T02:10:27.612860+0000","last_became_active":"2024-09-21T02:10:25.322895+0000","last_became_peered":"2024-09-21T02:10:25.322895+0000","last_unstale":"2024-09-21T02:10:27.612860+0000","last_undegraded":"2024-09-21T02:10:27.612860+0000","last_fullsized":"2024-09-21T02:10:27.612860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:43.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.287465+0000","last_change":"2024-09-21T02:10:26.334261+0000","last_active":"2024-09-21T02:10:28.287465+0000","last_peered":"2024-09-21T02:10:28.287465+0000","last_clean":"2024-09-21T02:10:28.287465+0000","last_became_active":"2024-09-21T02:10:26.333852+0000","last_became_peered":"2024-09-21T02:10:26.333852+0000","last_unstale":"2024-09-21T02:10:28.287465+0000","last_undegraded":"2024-09-21T02:10:28.287465+0000","last_fullsized":"2024-09-21T02:10:28.287465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:15.536953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286998+0000","last_change":"2024-09-21T02:10:25.299925+0000","last_active":"2024-09-21T02:10:27.286998+0000","last_peered":"2024-09-21T02:10:27.286998+0000","last_clean":"2024-09-21T02:10:27.286998+0000","last_became_active":"2024-09-21T02:10:25.299031+0000","last_became_peered":"2024-09-21T02:10:25.299031+0000","last_unstale":"2024-09-21T02:10:27.286998+0000","last_undegraded":"2024-09-21T02:10:27.286998+0000","last_fullsized":"2024-09-21T02:10:27.286998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:10.897487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.291929+0000","last_change":"2024-09-21T02:10:26.352265+0000","last_active":"2024-09-21T02:10:27.291929+0000","last_peered":"2024-09-21T02:10:27.291929+0000","last_clean":"2024-09-21T02:10:27.291929+0000","last_became_active":"2024-09-21T02:10:26.350994+0000","last_became_peered":"2024-09-21T02:10:26.350994+0000","last_unstale":"2024-09-21T02:10:27.291929+0000","last_undegraded":"2024-09-21T02:10:27.291929+0000","last_fullsized":"2024-09-21T02:10:27.291929+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:49.971661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294505+0000","last_change":"2024-09-21T02:10:25.312809+0000","last_active":"2024-09-21T02:10:28.294505+0000","last_peered":"2024-09-21T02:10:28.294505+0000","last_clean":"2024-09-21T02:10:28.294505+0000","last_became_active":"2024-09-21T02:10:25.310395+0000","last_became_peered":"2024-09-21T02:10:25.310395+0000","last_unstale":"2024-09-21T02:10:28.294505+0000","last_undegraded":"2024-09-21T02:10:28.294505+0000","last_fullsized":"2024-09-21T02:10:28.294505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:44.285547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872862+0000","last_change":"2024-09-21T02:10:26.300630+0000","last_active":"2024-09-21T02:10:27.872862+0000","last_peered":"2024-09-21T02:10:27.872862+0000","last_clean":"2024-09-21T02:10:27.872862+0000","last_became_active":"2024-09-21T02:10:26.300393+0000","last_became_peered":"2024-09-21T02:10:26.300393+0000","last_unstale":"2024-09-21T02:10:27.872862+0000","last_undegraded":"2024-09-21T02:10:27.872862+0000","last_fullsized":"2024-09-21T02:10:27.872862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:37.764592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.295144+0000","last_change":"2024-09-21T02:10:25.315653+0000","last_active":"2024-09-21T02:10:28.295144+0000","last_peered":"2024-09-21T02:10:28.295144+0000","last_clean":"2024-09-21T02:10:28.295144+0000","last_became_active":"2024-09-21T02:10:25.314901+0000","last_became_peered":"2024-09-21T02:10:25.314901+0000","last_unstale":"2024-09-21T02:10:28.295144+0000","last_undegraded":"2024-09-21T02:10:28.295144+0000","last_fullsized":"2024-09-21T02:10:28.295144+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:18.333180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.614562+0000","last_change":"2024-09-21T02:10:26.340943+0000","last_active":"2024-09-21T02:10:27.614562+0000","last_peered":"2024-09-21T02:10:27.614562+0000","last_clean":"2024-09-21T02:10:27.614562+0000","last_became_active":"2024-09-21T02:10:26.339737+0000","last_became_peered":"2024-09-21T02:10:26.339737+0000","last_unstale":"2024-09-21T02:10:27.614562+0000","last_undegraded":"2024-09-21T02:10:27.614562+0000","last_fullsized":"2024-09-21T02:10:27.614562+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.702073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286357+0000","last_change":"2024-09-21T02:10:25.315654+0000","last_active":"2024-09-21T02:10:27.286357+0000","last_peered":"2024-09-21T02:10:27.286357+0000","last_clean":"2024-09-21T02:10:27.286357+0000","last_became_active":"2024-09-21T02:10:25.314955+0000","last_became_peered":"2024-09-21T02:10:25.314955+0000","last_unstale":"2024-09-21T02:10:27.286357+0000","last_undegraded":"2024-09-21T02:10:27.286357+0000","last_fullsized":"2024-09-21T02:10:27.286357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:56.094584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872194+0000","last_change":"2024-09-21T02:10:26.343580+0000","last_active":"2024-09-21T02:10:27.872194+0000","last_peered":"2024-09-21T02:10:27.872194+0000","last_clean":"2024-09-21T02:10:27.872194+0000","last_became_active":"2024-09-21T02:10:26.343308+0000","last_became_peered":"2024-09-21T02:10:26.343308+0000","last_unstale":"2024-09-21T02:10:27.872194+0000","last_undegraded":"2024-09-21T02:10:27.872194+0000","last_fullsized":"2024-09-21T02:10:27.872194+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:13.702073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286233+0000","last_change":"2024-09-21T02:10:25.307587+0000","last_active":"2024-09-21T02:10:27.286233+0000","last_peered":"2024-09-21T02:10:27.286233+0000","last_clean":"2024-09-21T02:10:27.286233+0000","last_became_active":"2024-09-21T02:10:25.306663+0000","last_became_peered":"2024-09-21T02:10:25.306663+0000","last_unstale":"2024-09-21T02:10:27.286233+0000","last_undegraded":"2024-09-21T02:10:27.286233+0000","last_fullsized":"2024-09-21T02:10:27.286233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:35.066325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.285931+0000","last_change":"2024-09-21T02:10:26.307972+0000","last_active":"2024-09-21T02:10:27.285931+0000","last_peered":"2024-09-21T02:10:27.285931+0000","last_clean":"2024-09-21T02:10:27.285931+0000","last_became_active":"2024-09-21T02:10:26.307627+0000","last_became_peered":"2024-09-21T02:10:26.307627+0000","last_unstale":"2024-09-21T02:10:27.285931+0000","last_undegraded":"2024-09-21T02:10:27.285931+0000","last_fullsized":"2024-09-21T02:10:27.285931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:54.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599171+0000","last_change":"2024-09-21T02:10:25.310661+0000","last_active":"2024-09-21T02:10:27.599171+0000","last_peered":"2024-09-21T02:10:27.599171+0000","last_clean":"2024-09-21T02:10:27.599171+0000","last_became_active":"2024-09-21T02:10:25.310269+0000","last_became_peered":"2024-09-21T02:10:25.310269+0000","last_unstale":"2024-09-21T02:10:27.599171+0000","last_undegraded":"2024-09-21T02:10:27.599171+0000","last_fullsized":"2024-09-21T02:10:27.599171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:55.817406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290530+0000","last_change":"2024-09-21T02:10:26.316203+0000","last_active":"2024-09-21T02:10:27.290530+0000","last_peered":"2024-09-21T02:10:27.290530+0000","last_clean":"2024-09-21T02:10:27.290530+0000","last_became_active":"2024-09-21T02:10:26.314419+0000","last_became_peered":"2024-09-21T02:10:26.314419+0000","last_unstale":"2024-09-21T02:10:27.290530+0000","last_undegraded":"2024-09-21T02:10:27.290530+0000","last_fullsized":"2024-09-21T02:10:27.290530+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:23.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.290003+0000","last_change":"2024-09-21T02:10:25.331927+0000","last_active":"2024-09-21T02:10:27.290003+0000","last_peered":"2024-09-21T02:10:27.290003+0000","last_clean":"2024-09-21T02:10:27.290003+0000","last_became_active":"2024-09-21T02:10:25.331206+0000","last_became_peered":"2024-09-21T02:10:25.331206+0000","last_unstale":"2024-09-21T02:10:27.290003+0000","last_undegraded":"2024-09-21T02:10:27.290003+0000","last_fullsized":"2024-09-21T02:10:27.290003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:29.984913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286129+0000","last_change":"2024-09-21T02:10:26.308372+0000","last_active":"2024-09-21T02:10:27.286129+0000","last_peered":"2024-09-21T02:10:27.286129+0000","last_clean":"2024-09-21T02:10:27.286129+0000","last_became_active":"2024-09-21T02:10:26.307642+0000","last_became_peered":"2024-09-21T02:10:26.307642+0000","last_unstale":"2024-09-21T02:10:27.286129+0000","last_undegraded":"2024-09-21T02:10:27.286129+0000","last_fullsized":"2024-09-21T02:10:27.286129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:23.183644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.288506+0000","last_change":"2024-09-21T02:10:25.329725+0000","last_active":"2024-09-21T02:10:27.288506+0000","last_peered":"2024-09-21T02:10:27.288506+0000","last_clean":"2024-09-21T02:10:27.288506+0000","last_became_active":"2024-09-21T02:10:25.328986+0000","last_became_peered":"2024-09-21T02:10:25.328986+0000","last_unstale":"2024-09-21T02:10:27.288506+0000","last_undegraded":"2024-09-21T02:10:27.288506+0000","last_fullsized":"2024-09-21T02:10:27.288506+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:55.817406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286698+0000","last_change":"2024-09-21T02:10:26.328437+0000","last_active":"2024-09-21T02:10:27.286698+0000","last_peered":"2024-09-21T02:10:27.286698+0000","last_clean":"2024-09-21T02:10:27.286698+0000","last_became_active":"2024-09-21T02:10:26.328043+0000","last_became_peered":"2024-09-21T02:10:26.328043+0000","last_unstale":"2024-09-21T02:10:27.286698+0000","last_undegraded":"2024-09-21T02:10:27.286698+0000","last_fullsized":"2024-09-21T02:10:27.286698+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:27.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294389+0000","last_change":"2024-09-21T02:10:25.321560+0000","last_active":"2024-09-21T02:10:28.294389+0000","last_peered":"2024-09-21T02:10:28.294389+0000","last_clean":"2024-09-21T02:10:28.294389+0000","last_became_active":"2024-09-21T02:10:25.318778+0000","last_became_peered":"2024-09-21T02:10:25.318778+0000","last_unstale":"2024-09-21T02:10:28.294389+0000","last_undegraded":"2024-09-21T02:10:28.294389+0000","last_fullsized":"2024-09-21T02:10:28.294389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:59.645423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.295205+0000","last_change":"2024-09-21T02:10:26.321564+0000","last_active":"2024-09-21T02:10:28.295205+0000","last_peered":"2024-09-21T02:10:28.295205+0000","last_clean":"2024-09-21T02:10:28.295205+0000","last_became_active":"2024-09-21T02:10:26.320640+0000","last_became_peered":"2024-09-21T02:10:26.320640+0000","last_unstale":"2024-09-21T02:10:28.295205+0000","last_undegraded":"2024-09-21T02:10:28.295205+0000","last_fullsized":"2024-09-21T02:10:28.295205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:39.953694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.614396+0000","last_change":"2024-09-21T02:10:25.328416+0000","last_active":"2024-09-21T02:10:27.614396+0000","last_peered":"2024-09-21T02:10:27.614396+0000","last_clean":"2024-09-21T02:10:27.614396+0000","last_became_active":"2024-09-21T02:10:25.327630+0000","last_became_peered":"2024-09-21T02:10:25.327630+0000","last_unstale":"2024-09-21T02:10:27.614396+0000","last_undegraded":"2024-09-21T02:10:27.614396+0000","last_fullsized":"2024-09-21T02:10:27.614396+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:35.871538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.614310+0000","last_change":"2024-09-21T02:10:26.345022+0000","last_active":"2024-09-21T02:10:27.614310+0000","last_peered":"2024-09-21T02:10:27.614310+0000","last_clean":"2024-09-21T02:10:27.614310+0000","last_became_active":"2024-09-21T02:10:26.337446+0000","last_became_peered":"2024-09-21T02:10:26.337446+0000","last_unstale":"2024-09-21T02:10:27.614310+0000","last_undegraded":"2024-09-21T02:10:27.614310+0000","last_fullsized":"2024-09-21T02:10:27.614310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:57.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.286930+0000","last_change":"2024-09-21T02:10:25.314036+0000","last_active":"2024-09-21T02:10:28.286930+0000","last_peered":"2024-09-21T02:10:28.286930+0000","last_clean":"2024-09-21T02:10:28.286930+0000","last_became_active":"2024-09-21T02:10:25.313637+0000","last_became_peered":"2024-09-21T02:10:25.313637+0000","last_unstale":"2024-09-21T02:10:28.286930+0000","last_undegraded":"2024-09-21T02:10:28.286930+0000","last_fullsized":"2024-09-21T02:10:28.286930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:11.619326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.287759+0000","last_change":"2024-09-21T02:10:26.304346+0000","last_active":"2024-09-21T02:10:28.287759+0000","last_peered":"2024-09-21T02:10:28.287759+0000","last_clean":"2024-09-21T02:10:28.287759+0000","last_became_active":"2024-09-21T02:10:26.303656+0000","last_became_peered":"2024-09-21T02:10:26.303656+0000","last_unstale":"2024-09-21T02:10:28.287759+0000","last_undegraded":"2024-09-21T02:10:28.287759+0000","last_fullsized":"2024-09-21T02:10:28.287759+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:35.736941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599487+0000","last_change":"2024-09-21T02:10:25.316607+0000","last_active":"2024-09-21T02:10:27.599487+0000","last_peered":"2024-09-21T02:10:27.599487+0000","last_clean":"2024-09-21T02:10:27.599487+0000","last_became_active":"2024-09-21T02:10:25.316077+0000","last_became_peered":"2024-09-21T02:10:25.316077+0000","last_unstale":"2024-09-21T02:10:27.599487+0000","last_undegraded":"2024-09-21T02:10:27.599487+0000","last_fullsized":"2024-09-21T02:10:27.599487+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:47.362752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.289285+0000","last_change":"2024-09-21T02:10:26.326403+0000","last_active":"2024-09-21T02:10:27.289285+0000","last_peered":"2024-09-21T02:10:27.289285+0000","last_clean":"2024-09-21T02:10:27.289285+0000","last_became_active":"2024-09-21T02:10:26.318779+0000","last_became_peered":"2024-09-21T02:10:26.318779+0000","last_unstale":"2024-09-21T02:10:27.289285+0000","last_undegraded":"2024-09-21T02:10:27.289285+0000","last_fullsized":"2024-09-21T02:10:27.289285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:16.942625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.286819+0000","last_change":"2024-09-21T02:10:25.306491+0000","last_active":"2024-09-21T02:10:28.286819+0000","last_peered":"2024-09-21T02:10:28.286819+0000","last_clean":"2024-09-21T02:10:28.286819+0000","last_became_active":"2024-09-21T02:10:25.305970+0000","last_became_peered":"2024-09-21T02:10:25.305970+0000","last_unstale":"2024-09-21T02:10:28.286819+0000","last_undegraded":"2024-09-21T02:10:28.286819+0000","last_fullsized":"2024-09-21T02:10:28.286819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:54.249061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872071+0000","last_change":"2024-09-21T02:10:26.301720+0000","last_active":"2024-09-21T02:10:27.872071+0000","last_peered":"2024-09-21T02:10:27.872071+0000","last_clean":"2024-09-21T02:10:27.872071+0000","last_became_active":"2024-09-21T02:10:26.301424+0000","last_became_peered":"2024-09-21T02:10:26.301424+0000","last_unstale":"2024-09-21T02:10:27.872071+0000","last_undegraded":"2024-09-21T02:10:27.872071+0000","last_fullsized":"2024-09-21T02:10:27.872071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:54.670156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.286508+0000","last_change":"2024-09-21T02:10:25.320729+0000","last_active":"2024-09-21T02:10:28.286508+0000","last_peered":"2024-09-21T02:10:28.286508+0000","last_clean":"2024-09-21T02:10:28.286508+0000","last_became_active":"2024-09-21T02:10:25.320242+0000","last_became_peered":"2024-09-21T02:10:25.320242+0000","last_unstale":"2024-09-21T02:10:28.286508+0000","last_undegraded":"2024-09-21T02:10:28.286508+0000","last_fullsized":"2024-09-21T02:10:28.286508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:13.716034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294928+0000","last_change":"2024-09-21T02:10:26.348958+0000","last_active":"2024-09-21T02:10:28.294928+0000","last_peered":"2024-09-21T02:10:28.294928+0000","last_clean":"2024-09-21T02:10:28.294928+0000","last_became_active":"2024-09-21T02:10:26.348108+0000","last_became_peered":"2024-09-21T02:10:26.348108+0000","last_unstale":"2024-09-21T02:10:28.294928+0000","last_undegraded":"2024-09-21T02:10:28.294928+0000","last_fullsized":"2024-09-21T02:10:28.294928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:31.936778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.291092+0000","last_change":"2024-09-21T02:10:25.325201+0000","last_active":"2024-09-21T02:10:27.291092+0000","last_peered":"2024-09-21T02:10:27.291092+0000","last_clean":"2024-09-21T02:10:27.291092+0000","last_became_active":"2024-09-21T02:10:25.324553+0000","last_became_peered":"2024-09-21T02:10:25.324553+0000","last_unstale":"2024-09-21T02:10:27.291092+0000","last_undegraded":"2024-09-21T02:10:27.291092+0000","last_fullsized":"2024-09-21T02:10:27.291092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:47.362752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.286685+0000","last_change":"2024-09-21T02:10:26.308667+0000","last_active":"2024-09-21T02:10:27.286685+0000","last_peered":"2024-09-21T02:10:27.286685+0000","last_clean":"2024-09-21T02:10:27.286685+0000","last_became_active":"2024-09-21T02:10:26.308070+0000","last_became_peered":"2024-09-21T02:10:26.308070+0000","last_unstale":"2024-09-21T02:10:27.286685+0000","last_undegraded":"2024-09-21T02:10:27.286685+0000","last_fullsized":"2024-09-21T02:10:27.286685+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:52.593492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.286203+0000","last_change":"2024-09-21T02:10:25.305029+0000","last_active":"2024-09-21T02:10:28.286203+0000","last_peered":"2024-09-21T02:10:28.286203+0000","last_clean":"2024-09-21T02:10:28.286203+0000","last_became_active":"2024-09-21T02:10:25.304592+0000","last_became_peered":"2024-09-21T02:10:25.304592+0000","last_unstale":"2024-09-21T02:10:28.286203+0000","last_undegraded":"2024-09-21T02:10:28.286203+0000","last_fullsized":"2024-09-21T02:10:28.286203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:25.394885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599481+0000","last_change":"2024-09-21T02:10:26.349240+0000","last_active":"2024-09-21T02:10:27.599481+0000","last_peered":"2024-09-21T02:10:27.599481+0000","last_clean":"2024-09-21T02:10:27.599481+0000","last_became_active":"2024-09-21T02:10:26.348524+0000","last_became_peered":"2024-09-21T02:10:26.348524+0000","last_unstale":"2024-09-21T02:10:27.599481+0000","last_undegraded":"2024-09-21T02:10:27.599481+0000","last_fullsized":"2024-09-21T02:10:27.599481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:16.268178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294864+0000","last_change":"2024-09-21T02:10:25.308075+0000","last_active":"2024-09-21T02:10:28.294864+0000","last_peered":"2024-09-21T02:10:28.294864+0000","last_clean":"2024-09-21T02:10:28.294864+0000","last_became_active":"2024-09-21T02:10:25.307253+0000","last_became_peered":"2024-09-21T02:10:25.307253+0000","last_unstale":"2024-09-21T02:10:28.294864+0000","last_undegraded":"2024-09-21T02:10:28.294864+0000","last_fullsized":"2024-09-21T02:10:28.294864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:34.270274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.292557+0000","last_change":"2024-09-21T02:10:26.340281+0000","last_active":"2024-09-21T02:10:27.292557+0000","last_peered":"2024-09-21T02:10:27.292557+0000","last_clean":"2024-09-21T02:10:27.292557+0000","last_became_active":"2024-09-21T02:10:26.339554+0000","last_became_peered":"2024-09-21T02:10:26.339554+0000","last_unstale":"2024-09-21T02:10:27.292557+0000","last_undegraded":"2024-09-21T02:10:27.292557+0000","last_fullsized":"2024-09-21T02:10:27.292557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:56.483842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.287014+0000","last_change":"2024-09-21T02:10:25.302213+0000","last_active":"2024-09-21T02:10:28.287014+0000","last_peered":"2024-09-21T02:10:28.287014+0000","last_clean":"2024-09-21T02:10:28.287014+0000","last_became_active":"2024-09-21T02:10:25.301187+0000","last_became_peered":"2024-09-21T02:10:25.301187+0000","last_unstale":"2024-09-21T02:10:28.287014+0000","last_undegraded":"2024-09-21T02:10:28.287014+0000","last_fullsized":"2024-09-21T02:10:28.287014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:41.608357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.287239+0000","last_change":"2024-09-21T02:10:26.349243+0000","last_active":"2024-09-21T02:10:27.287239+0000","last_peered":"2024-09-21T02:10:27.287239+0000","last_clean":"2024-09-21T02:10:27.287239+0000","last_became_active":"2024-09-21T02:10:26.348869+0000","last_became_peered":"2024-09-21T02:10:26.348869+0000","last_unstale":"2024-09-21T02:10:27.287239+0000","last_undegraded":"2024-09-21T02:10:27.287239+0000","last_fullsized":"2024-09-21T02:10:27.287239+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:52.649914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.872206+0000","last_change":"2024-09-21T02:10:25.302467+0000","last_active":"2024-09-21T02:10:27.872206+0000","last_peered":"2024-09-21T02:10:27.872206+0000","last_clean":"2024-09-21T02:10:27.872206+0000","last_became_active":"2024-09-21T02:10:25.301945+0000","last_became_peered":"2024-09-21T02:10:25.301945+0000","last_unstale":"2024-09-21T02:10:27.872206+0000","last_undegraded":"2024-09-21T02:10:27.872206+0000","last_fullsized":"2024-09-21T02:10:27.872206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:17.257647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.873151+0000","last_change":"2024-09-21T02:10:26.302307+0000","last_active":"2024-09-21T02:10:27.873151+0000","last_peered":"2024-09-21T02:10:27.873151+0000","last_clean":"2024-09-21T02:10:27.873151+0000","last_became_active":"2024-09-21T02:10:26.301953+0000","last_became_peered":"2024-09-21T02:10:26.301953+0000","last_unstale":"2024-09-21T02:10:27.873151+0000","last_undegraded":"2024-09-21T02:10:27.873151+0000","last_fullsized":"2024-09-21T02:10:27.873151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:26.384129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.871985+0000","last_change":"2024-09-21T02:10:25.319104+0000","last_active":"2024-09-21T02:10:27.871985+0000","last_peered":"2024-09-21T02:10:27.871985+0000","last_clean":"2024-09-21T02:10:27.871985+0000","last_became_active":"2024-09-21T02:10:25.318728+0000","last_became_peered":"2024-09-21T02:10:25.318728+0000","last_unstale":"2024-09-21T02:10:27.871985+0000","last_undegraded":"2024-09-21T02:10:27.871985+0000","last_fullsized":"2024-09-21T02:10:27.871985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:12.129980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.599823+0000","last_change":"2024-09-21T02:10:26.327687+0000","last_active":"2024-09-21T02:10:27.599823+0000","last_peered":"2024-09-21T02:10:27.599823+0000","last_clean":"2024-09-21T02:10:27.599823+0000","last_became_active":"2024-09-21T02:10:26.327376+0000","last_became_peered":"2024-09-21T02:10:26.327376+0000","last_unstale":"2024-09-21T02:10:27.599823+0000","last_undegraded":"2024-09-21T02:10:27.599823+0000","last_fullsized":"2024-09-21T02:10:27.599823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.669217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.289282+0000","last_change":"2024-09-21T02:10:25.320871+0000","last_active":"2024-09-21T02:10:27.289282+0000","last_peered":"2024-09-21T02:10:27.289282+0000","last_clean":"2024-09-21T02:10:27.289282+0000","last_became_active":"2024-09-21T02:10:25.319866+0000","last_became_peered":"2024-09-21T02:10:25.319866+0000","last_unstale":"2024-09-21T02:10:27.289282+0000","last_undegraded":"2024-09-21T02:10:27.289282+0000","last_fullsized":"2024-09-21T02:10:27.289282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:34.030732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.613457+0000","last_change":"2024-09-21T02:10:26.351688+0000","last_active":"2024-09-21T02:10:27.613457+0000","last_peered":"2024-09-21T02:10:27.613457+0000","last_clean":"2024-09-21T02:10:27.613457+0000","last_became_active":"2024-09-21T02:10:26.341573+0000","last_became_peered":"2024-09-21T02:10:26.341573+0000","last_unstale":"2024-09-21T02:10:27.613457+0000","last_undegraded":"2024-09-21T02:10:27.613457+0000","last_fullsized":"2024-09-21T02:10:27.613457+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:00.649614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.612916+0000","last_change":"2024-09-21T02:10:25.302686+0000","last_active":"2024-09-21T02:10:27.612916+0000","last_peered":"2024-09-21T02:10:27.612916+0000","last_clean":"2024-09-21T02:10:27.612916+0000","last_became_active":"2024-09-21T02:10:25.301831+0000","last_became_peered":"2024-09-21T02:10:25.301831+0000","last_unstale":"2024-09-21T02:10:27.612916+0000","last_undegraded":"2024-09-21T02:10:27.612916+0000","last_fullsized":"2024-09-21T02:10:27.612916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:30.184608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.292634+0000","last_change":"2024-09-21T02:10:26.332612+0000","last_active":"2024-09-21T02:10:27.292634+0000","last_peered":"2024-09-21T02:10:27.292634+0000","last_clean":"2024-09-21T02:10:27.292634+0000","last_became_active":"2024-09-21T02:10:26.331561+0000","last_became_peered":"2024-09-21T02:10:26.331561+0000","last_unstale":"2024-09-21T02:10:27.292634+0000","last_undegraded":"2024-09-21T02:10:27.292634+0000","last_fullsized":"2024-09-21T02:10:27.292634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:51.897928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.285356+0000","last_change":"2024-09-21T02:10:25.299927+0000","last_active":"2024-09-21T02:10:27.285356+0000","last_peered":"2024-09-21T02:10:27.285356+0000","last_clean":"2024-09-21T02:10:27.285356+0000","last_became_active":"2024-09-21T02:10:25.299507+0000","last_became_peered":"2024-09-21T02:10:25.299507+0000","last_unstale":"2024-09-21T02:10:27.285356+0000","last_undegraded":"2024-09-21T02:10:27.285356+0000","last_fullsized":"2024-09-21T02:10:27.285356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:34.270274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:28.294632+0000","last_change":"2024-09-21T02:10:26.349131+0000","last_active":"2024-09-21T02:10:28.294632+0000","last_peered":"2024-09-21T02:10:28.294632+0000","last_clean":"2024-09-21T02:10:28.294632+0000","last_became_active":"2024-09-21T02:10:26.345071+0000","last_became_peered":"2024-09-21T02:10:26.345071+0000","last_unstale":"2024-09-21T02:10:28.294632+0000","last_undegraded":"2024-09-21T02:10:28.294632+0000","last_fullsized":"2024-09-21T02:10:28.294632+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:25.260288+0000","last_clean_scrub_stamp":"2024-09-21T02:10:25.260288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:51.345136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.291892+0000","last_change":"2024-09-21T02:10:25.320908+0000","last_active":"2024-09-21T02:10:27.291892+0000","last_peered":"2024-09-21T02:10:27.291892+0000","last_clean":"2024-09-21T02:10:27.291892+0000","last_became_active":"2024-09-21T02:10:25.316162+0000","last_became_peered":"2024-09-21T02:10:25.316162+0000","last_unstale":"2024-09-21T02:10:27.291892+0000","last_undegraded":"2024-09-21T02:10:27.291892+0000","last_fullsized":"2024-09-21T02:10:27.291892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:10:24.256097+0000","last_clean_scrub_stamp":"2024-09-21T02:10:24.256097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:12.763856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":147,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T02:10:27.288931+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:10:27.288931+0000","last_peered":"2024-09-21T02:10:27.288931+0000","last_clean":"2024-09-21T02:10:27.288931+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:10:27.288931+0000","last_undegraded":"2024-09-21T02:10:27.288931+0000","last_fullsized":"2024-09-21T02:10:27.288931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1069056,"data_stored":789912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":462848,"data_stored":198444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":198466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":475136,"data_stored":199508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":197932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":462848,"data_stored":197952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1056768,"data_stored":788264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:10:30.881 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:10:30.881 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 -- 172.21.15.96:0/2290380641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb2415ee60 msgr2=0x7fdb2415f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/2290380641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb2415ee60 0x7fdb2415f270 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb240d08f0 tx=0x7fdb0c01c830 comp rx=0 tx=0).stop 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 -- 172.21.15.96:0/2290380641 shutdown_connections 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/2290380641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdb2415b870 0x7fdb2415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/2290380641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb2415ee60 0x7fdb2415f270 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/2290380641 >> v2:172.21.15.122:3300/0 conn(0x7fdb24152300 0x7fdb240748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 -- 172.21.15.96:0/2290380641 >> 172.21.15.96:0/2290380641 conn(0x7fdb240d0490 msgr2=0x7fdb2406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 -- 172.21.15.96:0/2290380641 shutdown_connections 2024-09-21T02:10:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 -- 172.21.15.96:0/2290380641 wait complete. 2024-09-21T02:10:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 Processor -- start 2024-09-21T02:10:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.083+0000 7fdb2a20f640 1 -- start start 2024-09-21T02:10:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 0x7fdb24151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdb2415b870 0x7fdb241506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb2415ee60 0x7fdb24155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24070180 con 0x7fdb24152300 2024-09-21T02:10:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24070000 con 0x7fdb2415ee60 2024-09-21T02:10:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdb24070300 con 0x7fdb2415b870 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 0x7fdb24151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 0x7fdb24151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47006/0 (socket says 172.21.15.96:47006) 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 -- 172.21.15.96:0/3609621709 learned_addr learned my addr 172.21.15.96:0/3609621709 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb1affd640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdb2415b870 0x7fdb241506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb23fff640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb2415ee60 0x7fdb24155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 -- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdb2415b870 msgr2=0x7fdb241506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdb2415b870 0x7fdb241506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 -- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb2415ee60 msgr2=0x7fdb24155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb2415ee60 0x7fdb24155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 -- 172.21.15.96:0/3609621709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb241520b0 con 0x7fdb24152300 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb23fff640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb2415ee60 0x7fdb24155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:31.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb237fe640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 0x7fdb24151b70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400d6d0 tx=0x7fdb1400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14017070 con 0x7fdb24152300 2024-09-21T02:10:31.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb14009b10 con 0x7fdb24152300 2024-09-21T02:10:31.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241510d0 con 0x7fdb24152300 2024-09-21T02:10:31.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14010460 con 0x7fdb24152300 2024-09-21T02:10:31.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb24158ee0 con 0x7fdb24152300 2024-09-21T02:10:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2415bc80 con 0x7fdb24152300 2024-09-21T02:10:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdb1401a070 con 0x7fdb24152300 2024-09-21T02:10:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb20ff9640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdb04073200 0x7fdb04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdb14094550 con 0x7fdb24152300 2024-09-21T02:10:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.087+0000 7fdb1affd640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdb04073200 0x7fdb04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.091+0000 7fdb1affd640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdb04073200 0x7fdb04075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0af000 tx=0x7fdb0c01b0d0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.095+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdb1405f3f0 con 0x7fdb24152300 2024-09-21T02:10:31.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.319+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdb240cc450 con 0x7fdb24152300 2024-09-21T02:10:31.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb20ff9640 1 -- 172.21.15.96:0/3609621709 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) ==== 190+0+147 (secure 0 0 0) 0x7fdb1405ecf0 con 0x7fdb24152300 2024-09-21T02:10:31.328 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-09-21T02:10:31.328 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCEKu5my+CoJxAAog4sBSDU0v9/F60t9/U1nQ== 2024-09-21T02:10:31.328 INFO:teuthology.orchestra.run.smithi096.stdout: caps mds = "allow" 2024-09-21T02:10:31.328 INFO:teuthology.orchestra.run.smithi096.stdout: caps mon = "allow r" 2024-09-21T02:10:31.328 INFO:teuthology.orchestra.run.smithi096.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:10:31.329 INFO:teuthology.orchestra.run.smithi096.stderr:updated caps for client.0 2024-09-21T02:10:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdb04073200 msgr2=0x7fdb04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdb04073200 0x7fdb04075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0af000 tx=0x7fdb0c01b0d0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 msgr2=0x7fdb24151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 0x7fdb24151b70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400d6d0 tx=0x7fdb1400db90 comp rx=0 tx=0).stop 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 shutdown_connections 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdb04073200 0x7fdb04075710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb2415ee60 0x7fdb24155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdb2415b870 0x7fdb241506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 --2- 172.21.15.96:0/3609621709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb24152300 0x7fdb24151b70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 >> 172.21.15.96:0/3609621709 conn(0x7fdb240d0490 msgr2=0x7fdb24165530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 shutdown_connections 2024-09-21T02:10:31.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.323+0000 7fdb2a20f640 1 -- 172.21.15.96:0/3609621709 wait complete. 2024-09-21T02:10:31.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.495+0000 7f278d102640 1 -- 172.21.15.96:0/2293765590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 msgr2=0x7f2770005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.495+0000 7f278d102640 1 --2- 172.21.15.96:0/2293765590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f2770005200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2780001dc0 tx=0x7f278001cb30 comp rx=0 tx=0).stop 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.495+0000 7f278759e640 1 -- 172.21.15.96:0/2293765590 reap_dead start 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.495+0000 7f278d102640 1 -- 172.21.15.96:0/2293765590 shutdown_connections 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.495+0000 7f278d102640 1 -- 172.21.15.96:0/2293765590 >> 172.21.15.96:0/2293765590 conn(0x7f27880d1b10 msgr2=0x7f27880d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.499+0000 7f278d102640 1 -- 172.21.15.96:0/2293765590 shutdown_connections 2024-09-21T02:10:31.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.499+0000 7f278d102640 1 -- 172.21.15.96:0/2293765590 wait complete. 2024-09-21T02:10:31.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.499+0000 7f278d102640 1 Processor -- start 2024-09-21T02:10:31.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.499+0000 7f278d102640 1 -- start start 2024-09-21T02:10:31.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f27881529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f278814ede0 0x7f278814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27881454f0 0x7f27880b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f278815fbe0 con 0x7f2770004d50 2024-09-21T02:10:31.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27880cfc30 con 0x7f27881454f0 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f278815fd60 con 0x7f278814ede0 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f27881529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f27881529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47014/0 (socket says 172.21.15.96:47014) 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 -- 172.21.15.96:0/821612707 learned_addr learned my addr 172.21.15.96:0/821612707 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278659c640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f278814ede0 0x7f278814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278759e640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27881454f0 0x7f27880b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 -- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f278814ede0 msgr2=0x7f278814fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f278814ede0 0x7f278814fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 -- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27881454f0 msgr2=0x7f27880b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27881454f0 0x7f27880b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 -- 172.21.15.96:0/821612707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27881558b0 con 0x7f2770004d50 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278759e640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27881454f0 0x7f27880b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f2786d9d640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f27881529e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f277c00e940 tx=0x7f277c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c00ccc0 con 0x7f2770004d50 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f277c005020 con 0x7f2770004d50 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2788155b90 con 0x7f2770004d50 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c00ce60 con 0x7f2770004d50 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27881503b0 con 0x7f2770004d50 2024-09-21T02:10:31.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278814f460 con 0x7f2770004d50 2024-09-21T02:10:31.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f277c005450 con 0x7f2770004d50 2024-09-21T02:10:31.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f276f7fe640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2774073200 0x7f2774075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f277c0950f0 con 0x7f2770004d50 2024-09-21T02:10:31.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.503+0000 7f278659c640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2774073200 0x7f2774075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.507+0000 7f278659c640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2774073200 0x7f2774075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27780018b0 tx=0x7f2778001de0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.507+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f277c0600f0 con 0x7f2770004d50 2024-09-21T02:10:31.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.715+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2788069130 con 0x7f2770004d50 2024-09-21T02:10:31.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f276f7fe640 1 -- 172.21.15.96:0/821612707 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f277c05f9f0 con 0x7f2770004d50 2024-09-21T02:10:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:10:31.724 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"btime":"2024-09-21T02:10:29:323484+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:10:25.710385+0000","modified":"2024-09-21T02:10:28.731053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5037":{"gid":5037,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3851681791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3851681791},{"type":"v1","addr":"172.21.15.96:6835","nonce":3851681791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5037,"qdb_cluster":[5037]},"id":2}]} 2024-09-21T02:10:31.724 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2774073200 msgr2=0x7f2774075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2774073200 0x7f2774075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27780018b0 tx=0x7f2778001de0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 msgr2=0x7f27881529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f27881529e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f277c00e940 tx=0x7f277c00ee00 comp rx=0 tx=0).stop 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 shutdown_connections 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2774073200 0x7f2774075710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27881454f0 0x7f27880b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f278814ede0 0x7f278814fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 --2- 172.21.15.96:0/821612707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2770004d50 0x7f27881529e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 >> 172.21.15.96:0/821612707 conn(0x7f27880d1b10 msgr2=0x7f27880b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 shutdown_connections 2024-09-21T02:10:31.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:31.719+0000 7f278d102640 1 -- 172.21.15.96:0/821612707 wait complete. 2024-09-21T02:10:32.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:10:25.710385+0000', 'modified': '2024-09-21T02:10:28.731053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5037}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5037': {'gid': 5037, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3851681791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3851681791}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3851681791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5037, 'qdb_cluster': [5037]} 2024-09-21T02:10:32.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:10:32.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 -- 172.21.15.96:0/4126845916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ea8004030 msgr2=0x7f2ea8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 --2- 172.21.15.96:0/4126845916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ea8004030 0x7f2ea8004810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c610 tx=0x7f2eb401f6c0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ebed9d640 1 -- 172.21.15.96:0/4126845916 reap_dead start 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 -- 172.21.15.96:0/4126845916 shutdown_connections 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 -- 172.21.15.96:0/4126845916 >> 172.21.15.96:0/4126845916 conn(0x7f2ec006eea0 msgr2=0x7f2ec006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 -- 172.21.15.96:0/4126845916 shutdown_connections 2024-09-21T02:10:32.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 -- 172.21.15.96:0/4126845916 wait complete. 2024-09-21T02:10:32.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 Processor -- start 2024-09-21T02:10:32.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.311+0000 7f2ec4909640 1 -- start start 2024-09-21T02:10:32.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ec0151b10 0x7f2ec0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ec01596c0 0x7f2ec0159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 0x7f2ec01f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec00cbe80 con 0x7f2ec01f5820 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec00cbd00 con 0x7f2ec0151b10 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec00cc000 con 0x7f2ec01596c0 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 0x7f2ec01f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 0x7f2ec01f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47032/0 (socket says 172.21.15.96:47032) 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 -- 172.21.15.96:0/1448171062 learned_addr learned my addr 172.21.15.96:0/1448171062 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ec0151b10 0x7f2ec0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ec01596c0 0x7f2ec0159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 -- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ec01596c0 msgr2=0x7f2ec0159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ec01596c0 0x7f2ec0159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 -- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ec0151b10 msgr2=0x7f2ec0158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ec0151b10 0x7f2ec0158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 -- 172.21.15.96:0/1448171062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec00c47d0 con 0x7f2ec01f5820 2024-09-21T02:10:32.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 0x7f2ec01f5c30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00cca40 tx=0x7f2eb8010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2eaeffd640 1 -- 172.21.15.96:0/1448171062 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003d90 con 0x7f2ec01f5820 2024-09-21T02:10:32.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec00c4ab0 con 0x7f2ec01f5820 2024-09-21T02:10:32.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2eaeffd640 1 -- 172.21.15.96:0/1448171062 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb8015070 con 0x7f2ec01f5820 2024-09-21T02:10:32.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2eaeffd640 1 -- 172.21.15.96:0/1448171062 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800ec30 con 0x7f2ec01f5820 2024-09-21T02:10:32.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01580e0 con 0x7f2ec01f5820 2024-09-21T02:10:32.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.315+0000 7f2eaeffd640 1 -- 172.21.15.96:0/1448171062 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2eb8004030 con 0x7f2ec01f5820 2024-09-21T02:10:32.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2eaeffd640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2ea0073230 0x7f2ea0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2eaeffd640 1 -- 172.21.15.96:0/1448171062 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2eb8094a30 con 0x7f2ec01f5820 2024-09-21T02:10:32.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2ea0073230 0x7f2ea0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 Processor -- start 2024-09-21T02:10:32.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 -- start start 2024-09-21T02:10:32.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2ea0073230 0x7f2ea0075740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006fd0 tx=0x7f2eb0006b30 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c411d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c412250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c412790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d01c2e2410 con 0x55d01c2a5ca0 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d01c21fb40 con 0x55d01becc460 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55d01c20ef70 con 0x55d01c2bc950 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c412790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c412790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47038/0 (socket says 172.21.15.96:47038) 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 -- 172.21.15.96:0/397455082 learned_addr learned my addr 172.21.15.96:0/397455082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c412250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c411d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 -- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 msgr2=0x55d01c412250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c412250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 -- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 msgr2=0x55d01c411d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c411d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 -- 172.21.15.96:0/397455082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01c1d65b0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c412790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8000f90 tx=0x7f2eb805ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbf75f79eb776cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2e9ffff640 1 -- 172.21.15.96:0/397455082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8021070 con 0x55d01c2a5ca0 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2e9ffff640 1 -- 172.21.15.96:0/397455082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb8063d40 con 0x55d01c2a5ca0 2024-09-21T02:10:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2e9ffff640 1 -- 172.21.15.96:0/397455082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8068bf0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/397455082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 msgr2=0x55d01c412790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.319+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c412790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8000f90 tx=0x7f2eb805ed50 comp rx=0 tx=0).stop 2024-09-21T02:10:32.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/397455082 shutdown_connections 2024-09-21T02:10:32.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c412790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c412250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/397455082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c411d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/397455082 >> 172.21.15.96:0/397455082 conn(0x55d01c392d90 msgr2=0x55d01c3c2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/397455082 shutdown_connections 2024-09-21T02:10:32.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/397455082 wait complete. 2024-09-21T02:10:32.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- start start 2024-09-21T02:10:32.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 Processor -- start 2024-09-21T02:10:32.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 0x55d01c418f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2a5ca0 0x55d01c4194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c419a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d01c1a95f0 con 0x55d01becc460 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d01c212d80 con 0x55d01c2bc950 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55d01c1b5520 con 0x55d01c2a5ca0 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 10 client.0 _ll_register_callbacks cb 0x55d01c1b9830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 0x55d01c418f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 0x55d01c418f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47046/0 (socket says 172.21.15.96:47046) 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 -- 172.21.15.96:0/2517124808 learned_addr learned my addr 172.21.15.96:0/2517124808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2a5ca0 0x55d01c4194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c419a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 -- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2a5ca0 msgr2=0x55d01c4194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2a5ca0 0x55d01c4194d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 -- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 msgr2=0x55d01c419a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c419a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 -- 172.21.15.96:0/2517124808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d01c1ad190 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c419a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 0x55d01c418f90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0012a80 tx=0x7f2eb0001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66d911c6cf0e3c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2e9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/2517124808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0016070 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/2517124808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2eb0011d60 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d01c1a7110 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/2517124808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2eb002f050 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 10 client.5193 fetch_fsmap learned FSMap version 18 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55d01c293f10 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/2517124808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d01c1a7110 con 0x55d01becc460 2024-09-21T02:10:32.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/2517124808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f2eb0032020 con 0x55d01becc460 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 10 client.5193 fetch_fsmap finished waiting for FSMap version 18 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 10 client.5193 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 20 client.5193 populate_metadata read hostname 'smithi096' 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01beb17d0 0x55d01c423370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 4 client.5193 mds_command: new command op to 5037 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.323+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 --> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d01c221a90 con 0x55d01beb17d0 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01beb17d0 0x55d01c423370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2e9d7fa640 10 client.5193 ms_handle_connect on v2:172.21.15.96:6834/3851681791 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01beb17d0 0x55d01c423370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69376e53477c1129 server_cookie=3c246a3884e25ed2 in_seq=0 out_seq=0 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/2517124808 <== mds.0 v2:172.21.15.96:6834/3851681791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d01c221a90 con 0x55d01beb17d0 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2e9d7fa640 10 client.5193 handle_command_reply: tid=0 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 client.5193 shutdown 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 msgr2=0x55d01c418f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 0x55d01c418f90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0012a80 tx=0x7f2eb0001130 comp rx=0 tx=0).stop 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 shutdown_connections 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01beb17d0 0x55d01c423370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c419a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2a5ca0 0x55d01c4194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2e9d7fa640 0 client.5193 ms_handle_reset on v2:172.21.15.96:6834/3851681791 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2517124808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01becc460 0x55d01c418f90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 >> 172.21.15.96:0/2517124808 conn(0x55d01c392d90 msgr2=0x55d01c3c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 shutdown_connections 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2517124808 wait complete. 2024-09-21T02:10:32.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.327+0000 7f2ecbbd4000 20 client.5193 trim_cache size 0 max 16384 2024-09-21T02:10:32.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 Processor -- start 2024-09-21T02:10:32.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- start start 2024-09-21T02:10:32.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c4834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c483a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c483f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d01c3d1790 con 0x55d01c2a5ca0 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01fecf0 con 0x55d01becc460 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec01fdd40 con 0x55d01c2bc950 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c483a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c483a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47052/0 (socket says 172.21.15.96:47052) 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/2833298131 learned_addr learned my addr 172.21.15.96:0/2833298131 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c483f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c4834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 msgr2=0x55d01c483f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c483f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 msgr2=0x55d01c4834d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c4834d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/2833298131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01c26adb0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c483a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000c10 tx=0x7f2eb401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d303e0d37b2ea093 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2e9f7fe640 1 -- 172.21.15.96:0/2833298131 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40b3040 con 0x55d01c2a5ca0 2024-09-21T02:10:32.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2e9f7fe640 1 -- 172.21.15.96:0/2833298131 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb40c0030 con 0x55d01c2a5ca0 2024-09-21T02:10:32.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2e9f7fe640 1 -- 172.21.15.96:0/2833298131 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40018d0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2833298131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 msgr2=0x55d01c483a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c483a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000c10 tx=0x7f2eb401c500 comp rx=0 tx=0).stop 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2833298131 shutdown_connections 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01c2bc950 0x55d01c483f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c483a10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/2833298131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01becc460 0x55d01c4834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2833298131 >> 172.21.15.96:0/2833298131 conn(0x55d01c392d90 msgr2=0x55d01c3cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2833298131 shutdown_connections 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.347+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/2833298131 wait complete. 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- start start 2024-09-21T02:10:32.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 Processor -- start 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01becc460 0x55d01c426410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c426950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c443af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d01c1a95f0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d01c1daee0 con 0x55d01c2bc950 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55d01c2173b0 con 0x55d01becc460 2024-09-21T02:10:32.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 10 client.0 _ll_register_callbacks cb 0x7f2eb00062d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c426950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c426950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47066/0 (socket says 172.21.15.96:47066) 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/1369299604 learned_addr learned my addr 172.21.15.96:0/1369299604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01becc460 0x55d01c426410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebed9d640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c443af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01becc460 msgr2=0x55d01c426410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01becc460 0x55d01c426410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 msgr2=0x55d01c443af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c443af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 -- 172.21.15.96:0/1369299604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d01c1d5ab0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebdd9b640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c426950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4001ec0 tx=0x7f2eb40069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c72c283f2e9b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2e9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-21T02:10:32.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/1369299604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40b3030 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/1369299604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2eb40060b0 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb40c0030 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/1369299604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2eb40d0050 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 10 client.5199 fetch_fsmap learned FSMap version 18 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55d01c1b4070 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/1369299604 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb40c0030 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/1369299604 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f2eb40d3020 con 0x55d01c2a5ca0 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 10 client.5199 fetch_fsmap finished waiting for FSMap version 18 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 10 client.5199 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 20 client.5199 populate_metadata read hostname 'smithi096' 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01be9f830 0x55d01c444480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 4 client.5199 mds_command: new command op to 5037 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:10:32.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 --> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] -- command(tid 0: {"prefix": "status"}) -- 0x55d01c21d540 con 0x55d01be9f830 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.351+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01be9f830 0x55d01c444480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2e9d7fa640 10 client.5199 ms_handle_connect on v2:172.21.15.96:6834/3851681791 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ebe59c640 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01be9f830 0x55d01c444480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b543f0d3a9e29ec2 server_cookie=1ec8c9aea53f3ad1 in_seq=0 out_seq=0 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2e9d7fa640 1 -- 172.21.15.96:0/1369299604 <== mds.0 v2:172.21.15.96:6834/3851681791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55d01c21d540 con 0x55d01be9f830 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2e9d7fa640 10 client.5199 handle_command_reply: tid=0 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 client.5199 shutdown 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 msgr2=0x55d01c426950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c426950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4001ec0 tx=0x7f2eb40069b0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 shutdown_connections 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:6834/3851681791,v1:172.21.15.96:6835/3851681791] conn(0x55d01be9f830 0x55d01c444480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d01c2bc950 0x55d01c443af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2e9d7fa640 0 client.5199 ms_handle_reset on v2:172.21.15.96:6834/3851681791 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d01c2a5ca0 0x55d01c426950 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 --2- 172.21.15.96:0/1369299604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55d01becc460 0x55d01c426410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 >> 172.21.15.96:0/1369299604 conn(0x55d01c392d90 msgr2=0x55d01c3cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 shutdown_connections 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 1 -- 172.21.15.96:0/1369299604 wait complete. 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ecbbd4000 20 client.5199 trim_cache size 0 max 16384 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "892bf3fa-0a6f-4242-b0ca-536a4d478630", 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5037, 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-21T02:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 6.0795621640000004, 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 18, 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 40, 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 40, 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 13.43569961 2024-09-21T02:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2ea0073230 msgr2=0x7f2ea0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2ea0073230 0x7f2ea0075740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006fd0 tx=0x7f2eb0006b30 comp rx=0 tx=0).stop 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 msgr2=0x7f2ec01f5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 0x7f2ec01f5c30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00cca40 tx=0x7f2eb8010dd0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 shutdown_connections 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2ea0073230 0x7f2ea0075740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec01f5820 0x7f2ec01f5c30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ec01596c0 0x7f2ec0159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 --2- 172.21.15.96:0/1448171062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ec0151b10 0x7f2ec0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 >> 172.21.15.96:0/1448171062 conn(0x7f2ec006eea0 msgr2=0x7f2ec0152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 shutdown_connections 2024-09-21T02:10:32.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.355+0000 7f2ec4909640 1 -- 172.21.15.96:0/1448171062 wait complete. 2024-09-21T02:10:32.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 -- 172.21.15.96:0/1428046345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac0cf710 msgr2=0x7f9eac15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/1428046345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac0cf710 0x7f9eac15f290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009840 tx=0x7f9e9c01c820 comp rx=0 tx=0).stop 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 -- 172.21.15.96:0/1428046345 shutdown_connections 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/1428046345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac0cf710 0x7f9eac15f290 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 -- 172.21.15.96:0/1428046345 >> 172.21.15.96:0/1428046345 conn(0x7f9eac0d1b10 msgr2=0x7f9eac0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 -- 172.21.15.96:0/1428046345 shutdown_connections 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 -- 172.21.15.96:0/1428046345 wait complete. 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 Processor -- start 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.519+0000 7f9eb09ee640 1 -- start start 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eac0cf710 0x7f9eac0ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9eac0d3470 0x7f9eac0baca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 0x7f9eac15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac16cf40 con 0x7f9eac15f7d0 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac16cdc0 con 0x7f9eac0cf710 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac16d0c0 con 0x7f9eac0d3470 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 0x7f9eac15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 0x7f9eac15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47080/0 (socket says 172.21.15.96:47080) 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 -- 172.21.15.96:0/144120539 learned_addr learned my addr 172.21.15.96:0/144120539 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaa59c640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eac0cf710 0x7f9eac0ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9ea9d9b640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9eac0d3470 0x7f9eac0baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 -- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9eac0d3470 msgr2=0x7f9eac0baca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9eac0d3470 0x7f9eac0baca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 -- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eac0cf710 msgr2=0x7f9eac0ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eac0cf710 0x7f9eac0ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 -- 172.21.15.96:0/144120539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac149fd0 con 0x7f9eac15f7d0 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eaad9d640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 0x7f9eac15b7f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c6b0 tx=0x7f9ea400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4003b80 con 0x7f9eac15f7d0 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ea4003d20 con 0x7f9eac15f7d0 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac14a2b0 con 0x7f9eac15f7d0 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4005330 con 0x7f9eac15f7d0 2024-09-21T02:10:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac14a870 con 0x7f9eac15f7d0 2024-09-21T02:10:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.523+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac15dd20 con 0x7f9eac15f7d0 2024-09-21T02:10:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.527+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ea40054d0 con 0x7f9eac15f7d0 2024-09-21T02:10:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.527+0000 7f9e9affd640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9e8c073200 0x7f9e8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.527+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9ea4094b50 con 0x7f9eac15f7d0 2024-09-21T02:10:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.527+0000 7f9eaa59c640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9e8c073200 0x7f9e8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.527+0000 7f9eaa59c640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9e8c073200 0x7f9e8c075710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0af000 tx=0x7f9e9c01b060 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.531+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9ea405fb50 con 0x7f9eac15f7d0 2024-09-21T02:10:32.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9eac149d60 con 0x7f9eac15f7d0 2024-09-21T02:10:32.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9e9affd640 1 -- 172.21.15.96:0/144120539 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f9ea405f450 con 0x7f9eac15f7d0 2024-09-21T02:10:32.732 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:10:32.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9e8c073200 msgr2=0x7f9e8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9e8c073200 0x7f9e8c075710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0af000 tx=0x7f9e9c01b060 comp rx=0 tx=0).stop 2024-09-21T02:10:32.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 msgr2=0x7f9eac15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 0x7f9eac15b7f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c6b0 tx=0x7f9ea400cb70 comp rx=0 tx=0).stop 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 shutdown_connections 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9e8c073200 0x7f9e8c075710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9eac15f7d0 0x7f9eac15b7f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9eac0d3470 0x7f9eac0baca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 --2- 172.21.15.96:0/144120539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eac0cf710 0x7f9eac0ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 >> 172.21.15.96:0/144120539 conn(0x7f9eac0d1b10 msgr2=0x7f9eac15e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 shutdown_connections 2024-09-21T02:10:32.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32.727+0000 7f9eb09ee640 1 -- 172.21.15.96:0/144120539 wait complete. 2024-09-21T02:10:32.751 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:10:32.751 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:10:32.752 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:10:32.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:32.752 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:10:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:e1:97:6e:59 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:10:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:10:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:10:32.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:32.761 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:10:32.806 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-21T02:10:32.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:32.807 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-21T02:10:32.863 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-21T02:10:32.863 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:10:32.863 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:10:32.864 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:10:32.864 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-21T02:10:32.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:10:32.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:10:32.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:10:32.865 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-21T02:10:32.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:10:32.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:10:32.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:32.987 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-21T02:10:33.035 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:10:33.035 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:10:33.035 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:10:33.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:33.035 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-21T02:10:33.093 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:10:33.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:10:33.093 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:33.139 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:10:33.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:10:33.140 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:33.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:10:33.244 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:10:33.293 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:10:33.294 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:33.295 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:33.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:33.296 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:10:33.343 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:10:33.343 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:10:33.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:10:33.401 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:10:33.450 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:10:33.451 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:33.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:33.453 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:10:33.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-21T02:10:33.487+0000 7ff53e36f580 -1 init, newargv = 0x557545bc6670 newargc=15 2024-09-21T02:10:33.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[38609]: starting ceph client 2024-09-21T02:10:33.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[38609]: starting fuse 2024-09-21T02:10:34.598 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-21T02:10:34.614 DEBUG:teuthology.orchestra.run.smithi096:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:10:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:/sys 2024-09-21T02:10:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:/proc 2024-09-21T02:10:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:/dev 2024-09-21T02:10:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/pts 2024-09-21T02:10:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:/run 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/ 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/security 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/shm 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/run/lock 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/cgroup 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/pstore 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/bpf 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/hugepages 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/mqueue 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/debug 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/tracing 2024-09-21T02:10:34.661 INFO:teuthology.orchestra.run.smithi096.stdout:/proc/fs/nfsd 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/fs/fuse/connections 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/sys/kernel/config 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/run/rpc_pipefs 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/run/user/1000 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:34.662 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:10:34.663 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:34.663 INFO:teuthology.orchestra.run.smithi096.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:10:34.663 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:34.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:34.664 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-21T02:10:34.711 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-21T02:10:34.712 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-21T02:10:34.712 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-21T02:10:34.713 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-21T02:10:34.714 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:10:34.714 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-21T02:10:34.714 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-21T02:10:34.714 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-21T02:10:34.834 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.38609.asok 2024-09-21T02:10:34.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:10:34.846 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38609.asok 2024-09-21T02:10:34.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:34.846 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38609.asok status 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "38609", 2024-09-21T02:10:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5214, 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 5214 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2967381039 2024-09-21T02:10:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2967381039 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.5214 192.168.0.1:0/2967381039", 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/2967381039", 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 18, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 40, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-21T02:10:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-21T02:10:35.022 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:10:35.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:35.035 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-21T02:10:35.035 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:35.041 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-21T02:10:35.041 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:35.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:35.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:10:35.095 DEBUG:teuthology.orchestra.run.smithi096:> mktemp -d --suffix=xfstests-dev 2024-09-21T02:10:35.139 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:10:35.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:10:35.143 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.trp2hAHaMUxfstests-dev 2024-09-21T02:10:35.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:35.143 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.trp2hAHaMUxfstests-dev) 2024-09-21T02:10:35.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:35.198 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.trp2hAHaMUxfstests-dev) 2024-09-21T02:10:35.257 INFO:teuthology.orchestra.run.smithi096.stderr:Cloning into '/tmp/tmp.trp2hAHaMUxfstests-dev'... 2024-09-21T02:10:35.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:35.890 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-09-21T02:10:35.946 DEBUG:teuthology.orchestra.run.smithi096:> mktemp -d --suffix=test 2024-09-21T02:10:35.995 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.cAh74ajS05test 2024-09-21T02:10:35.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:10:35.995 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-09-21T02:10:36.049 DEBUG:teuthology.orchestra.run.smithi096:> mktemp -d --suffix=scratch 2024-09-21T02:10:36.095 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.i5J4KxFJ38scratch 2024-09-21T02:10:36.095 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "ubuntu" and "22.04". 2024-09-21T02:10:36.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc git indent libacl1-dev libaio-dev libcap-dev libgdbm-dev libtool libtool-bin liburing-dev libuuid1 lvm2 make psmisc python3 quota sed uuid-dev uuid-runtime xfsprogs sqlite3 exfatprogs f2fs-tools ocfs2-tools udftools xfsdump xfslibs-dev 2024-09-21T02:10:36.208 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-21T02:10:36.426 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-21T02:10:36.426 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-21T02:10:36.645 INFO:teuthology.orchestra.run.smithi096.stdout:acl is already the newest version (2.3.1-1). 2024-09-21T02:10:36.645 INFO:teuthology.orchestra.run.smithi096.stdout:acl set to manually installed. 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:attr is already the newest version (1:2.5.1-1build1). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:automake is already the newest version (1:1.16.5-1.3). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:bc is already the newest version (1.07.1-3build1). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:gcc is already the newest version (4:11.2.0-1ubuntu1). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:gcc set to manually installed. 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:libacl1-dev is already the newest version (2.3.1-1). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:libaio-dev is already the newest version (0.3.112-13build1). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:libtool is already the newest version (2.4.6-15build2). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:lvm2 is already the newest version (2.03.11-2.1ubuntu4). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:make is already the newest version (4.3-4.1build1). 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:make set to manually installed. 2024-09-21T02:10:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:psmisc is already the newest version (23.4-2build3). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:quota is already the newest version (4.06-1build2). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:sed is already the newest version (4.8-1ubuntu2). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:xfsdump is already the newest version (3.1.9+0+nmu2). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:xfsprogs is already the newest version (5.13.0-1ubuntu2). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:dbench is already the newest version (4.0-2build1). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:dump is already the newest version (0.4b47-3). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:git is already the newest version (1:2.34.1-1ubuntu1.11). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:indent is already the newest version (2.2.12-1ubuntu0.22.04.1). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:libuuid1 is already the newest version (2.37.2-4ubuntu3.4). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3.4). 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:10:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libiscsi7 libpmemobj1 2024-09-21T02:10:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev 2024-09-21T02:10:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7-minimal libpython2.7-stdlib linux-image-5.4.0-135-generic 2024-09-21T02:10:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-5.4.0-135-generic python2 python2-dev python2-minimal 2024-09-21T02:10:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: libcap2 libcmap4 libcorosync-common4 libdlm3 libext2fs2 libpmemblk1 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: libpython3-dev libpython3-stdlib libqb100 libsqlite3-0 python3-dev 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: python3-minimal python3-venv 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: gpart fuse2fs e2fsck-static gnuplot gfio python-scipy gawk-doc python3-doc 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: sqlite3-doc dvd+rw-tools udfclient 2024-09-21T02:10:36.649 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-21T02:10:36.650 INFO:teuthology.orchestra.run.smithi096.stdout: e2fsprogs-l10n 2024-09-21T02:10:36.688 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-21T02:10:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: exfatprogs f2fs-tools fio libcap-dev libcmap4 libcorosync-common4 libdlm3 2024-09-21T02:10:36.689 INFO:teuthology.orchestra.run.smithi096.stdout: libgdbm-dev libpmemblk1 libqb100 liburing-dev ocfs2-tools sqlite3 udftools 2024-09-21T02:10:36.690 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-09-21T02:10:36.690 INFO:teuthology.orchestra.run.smithi096.stdout: e2fsprogs gawk libcap2 libext2fs2 libpython3-dev libpython3-stdlib 2024-09-21T02:10:36.691 INFO:teuthology.orchestra.run.smithi096.stdout: libsqlite3-0 python3 python3-dev python3-minimal python3-venv uuid-runtime 2024-09-21T02:10:36.762 INFO:teuthology.orchestra.run.smithi096.stdout:12 upgraded, 14 newly installed, 0 to remove and 316 not upgraded. 2024-09-21T02:10:36.762 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 8,201 kB of archives. 2024-09-21T02:10:36.762 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 14.2 MB of additional disk space will be used. 2024-09-21T02:10:36.762 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libext2fs2 amd64 1.46.5-2ubuntu1.2 [208 kB] 2024-09-21T02:10:36.857 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 e2fsprogs amd64 1.46.5-2ubuntu1.2 [590 kB] 2024-09-21T02:10:36.895 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-venv amd64 3.10.6-1~22.04.1 [1,042 B] 2024-09-21T02:10:36.895 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpython3-dev amd64 3.10.6-1~22.04.1 [7,064 B] 2024-09-21T02:10:36.896 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-dev amd64 3.10.6-1~22.04.1 [26.0 kB] 2024-09-21T02:10:36.897 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-minimal amd64 3.10.6-1~22.04.1 [24.3 kB] 2024-09-21T02:10:36.898 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3 amd64 3.10.6-1~22.04.1 [22.8 kB] 2024-09-21T02:10:36.899 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpython3-stdlib amd64 3.10.6-1~22.04.1 [6,812 B] 2024-09-21T02:10:36.900 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-runtime amd64 2.37.2-4ubuntu3.4 [32.0 kB] 2024-09-21T02:10:36.901 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 gawk amd64 1:5.1.0-1ubuntu0.1 [447 kB] 2024-09-21T02:10:36.911 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcorosync-common4 amd64 3.1.6-1ubuntu1 [7,322 B] 2024-09-21T02:10:36.913 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqb100 amd64 2.0.4-1ubuntu0.2 [74.2 kB] 2024-09-21T02:10:36.929 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcmap4 amd64 3.1.6-1ubuntu1 [10.4 kB] 2024-09-21T02:10:36.930 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libdlm3 amd64 4.1.1-1build1 [14.4 kB] 2024-09-21T02:10:36.931 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 ocfs2-tools amd64 1.8.7-1build1 [646 kB] 2024-09-21T02:10:36.946 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap2 amd64 1:2.44-1ubuntu0.22.04.1 [18.3 kB] 2024-09-21T02:10:36.946 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-09-21T02:10:36.961 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 exfatprogs amd64 1.1.3-1ubuntu0.1 [40.5 kB] 2024-09-21T02:10:36.962 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 f2fs-tools amd64 1.14.0-2build1 [169 kB] 2024-09-21T02:10:36.965 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libpmemblk1 amd64 1.11.1-3build1 [65.6 kB] 2024-09-21T02:10:36.967 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 fio amd64 3.28-1 [4,047 kB] 2024-09-21T02:10:37.015 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap-dev amd64 1:2.44-1ubuntu0.22.04.1 [39.4 kB] 2024-09-21T02:10:37.015 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgdbm-dev amd64 1.23-1 [117 kB] 2024-09-21T02:10:37.017 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liburing-dev amd64 2.1-2build1 [51.8 kB] 2024-09-21T02:10:37.017 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-09-21T02:10:37.024 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 udftools amd64 2.3-1 [125 kB] 2024-09-21T02:10:37.305 INFO:teuthology.orchestra.run.smithi096.stderr:debconf: unable to initialize frontend: Dialog 2024-09-21T02:10:37.305 INFO:teuthology.orchestra.run.smithi096.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-09-21T02:10:37.306 INFO:teuthology.orchestra.run.smithi096.stderr:debconf: falling back to frontend: Readline 2024-09-21T02:10:37.321 INFO:teuthology.orchestra.run.smithi096.stderr:debconf: unable to initialize frontend: Readline 2024-09-21T02:10:37.321 INFO:teuthology.orchestra.run.smithi096.stderr:debconf: (This frontend requires a controlling tty.) 2024-09-21T02:10:37.321 INFO:teuthology.orchestra.run.smithi096.stderr:debconf: falling back to frontend: Teletype 2024-09-21T02:10:37.325 INFO:teuthology.orchestra.run.smithi096.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-09-21T02:10:37.358 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 8,201 kB in 0s (25.1 MB/s) 2024-09-21T02:10:37.641 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 179161 files and directories currently installed.) 2024-09-21T02:10:37.646 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libext2fs2_1.46.5-2ubuntu1.2_amd64.deb ... 2024-09-21T02:10:37.814 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libext2fs2:amd64 (1.46.5-2ubuntu1.2) over (1.46.5-2ubuntu1.1) ... 2024-09-21T02:10:38.166 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libext2fs2:amd64 (1.46.5-2ubuntu1.2) ... 2024-09-21T02:10:38.723 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 179161 files and directories currently installed.) 2024-09-21T02:10:38.729 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../e2fsprogs_1.46.5-2ubuntu1.2_amd64.deb ... 2024-09-21T02:10:38.883 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking e2fsprogs (1.46.5-2ubuntu1.2) over (1.46.5-2ubuntu1.1) ... 2024-09-21T02:10:39.290 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../python3-venv_3.10.6-1~22.04.1_amd64.deb ... 2024-09-21T02:10:39.448 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-venv (3.10.6-1~22.04.1) over (3.10.6-1~22.04) ... 2024-09-21T02:10:39.760 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libpython3-dev_3.10.6-1~22.04.1_amd64.deb ... 2024-09-21T02:10:39.910 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpython3-dev:amd64 (3.10.6-1~22.04.1) over (3.10.6-1~22.04) ... 2024-09-21T02:10:40.231 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../python3-dev_3.10.6-1~22.04.1_amd64.deb ... 2024-09-21T02:10:40.357 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-dev (3.10.6-1~22.04.1) over (3.10.6-1~22.04) ... 2024-09-21T02:10:40.832 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../python3-minimal_3.10.6-1~22.04.1_amd64.deb ... 2024-09-21T02:10:41.004 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-minimal (3.10.6-1~22.04.1) over (3.10.6-1~22.04) ... 2024-09-21T02:10:41.359 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-minimal (3.10.6-1~22.04.1) ... 2024-09-21T02:10:42.031 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 179161 files and directories currently installed.) 2024-09-21T02:10:42.037 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-python3_3.10.6-1~22.04.1_amd64.deb ... 2024-09-21T02:10:42.255 INFO:teuthology.orchestra.run.smithi096.stdout:running python pre-rtupdate hooks for python3.10... 2024-09-21T02:10:42.259 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3 (3.10.6-1~22.04.1) over (3.10.6-1~22.04) ... 2024-09-21T02:10:42.567 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-libpython3-stdlib_3.10.6-1~22.04.1_amd64.deb ... 2024-09-21T02:10:42.688 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpython3-stdlib:amd64 (3.10.6-1~22.04.1) over (3.10.6-1~22.04) ... 2024-09-21T02:10:43.055 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-uuid-runtime_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:10:43.184 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking uuid-runtime (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:10:43.594 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-gawk_1%3a5.1.0-1ubuntu0.1_amd64.deb ... 2024-09-21T02:10:43.724 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking gawk (1:5.1.0-1ubuntu0.1) over (1:5.1.0-1build3) ... 2024-09-21T02:10:44.219 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcorosync-common4:amd64. 2024-09-21T02:10:44.248 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-libcorosync-common4_3.1.6-1ubuntu1_amd64.deb ... 2024-09-21T02:10:44.285 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-09-21T02:10:44.630 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqb100:amd64. 2024-09-21T02:10:44.660 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-libqb100_2.0.4-1ubuntu0.2_amd64.deb ... 2024-09-21T02:10:44.697 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-09-21T02:10:45.009 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcmap4:amd64. 2024-09-21T02:10:45.037 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../6-libcmap4_3.1.6-1ubuntu1_amd64.deb ... 2024-09-21T02:10:45.075 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-09-21T02:10:45.403 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libdlm3:amd64. 2024-09-21T02:10:45.432 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../7-libdlm3_4.1.1-1build1_amd64.deb ... 2024-09-21T02:10:45.470 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libdlm3:amd64 (4.1.1-1build1) ... 2024-09-21T02:10:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ocfs2-tools. 2024-09-21T02:10:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../8-ocfs2-tools_1.8.7-1build1_amd64.deb ... 2024-09-21T02:10:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ocfs2-tools (1.8.7-1build1) ... 2024-09-21T02:10:46.300 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../9-libcap2_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:10:46.438 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) over (1:2.44-1build3) ... 2024-09-21T02:10:46.792 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-09-21T02:10:47.271 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 179235 files and directories currently installed.) 2024-09-21T02:10:47.277 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-09-21T02:10:47.405 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-09-21T02:10:47.735 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package exfatprogs. 2024-09-21T02:10:47.752 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-exfatprogs_1.1.3-1ubuntu0.1_amd64.deb ... 2024-09-21T02:10:47.793 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking exfatprogs (1.1.3-1ubuntu0.1) ... 2024-09-21T02:10:48.113 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package f2fs-tools. 2024-09-21T02:10:48.142 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-f2fs-tools_1.14.0-2build1_amd64.deb ... 2024-09-21T02:10:48.230 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking f2fs-tools (1.14.0-2build1) ... 2024-09-21T02:10:48.601 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpmemblk1:amd64. 2024-09-21T02:10:48.630 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-libpmemblk1_1.11.1-3build1_amd64.deb ... 2024-09-21T02:10:48.668 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpmemblk1:amd64 (1.11.1-3build1) ... 2024-09-21T02:10:49.004 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package fio. 2024-09-21T02:10:49.033 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-fio_3.28-1_amd64.deb ... 2024-09-21T02:10:49.071 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking fio (3.28-1) ... 2024-09-21T02:10:49.492 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcap-dev:amd64. 2024-09-21T02:10:49.509 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-libcap-dev_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:10:49.542 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-09-21T02:10:49.814 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libgdbm-dev:amd64. 2024-09-21T02:10:49.843 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../6-libgdbm-dev_1.23-1_amd64.deb ... 2024-09-21T02:10:49.880 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libgdbm-dev:amd64 (1.23-1) ... 2024-09-21T02:10:50.201 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liburing-dev:amd64. 2024-09-21T02:10:50.228 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../7-liburing-dev_2.1-2build1_amd64.deb ... 2024-09-21T02:10:50.267 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liburing-dev:amd64 (2.1-2build1) ... 2024-09-21T02:10:50.496 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package sqlite3. 2024-09-21T02:10:50.511 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../8-sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-09-21T02:10:50.554 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-09-21T02:10:51.558 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package udftools. 2024-09-21T02:10:51.587 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../9-udftools_2.3-1_amd64.deb ... 2024-09-21T02:10:51.644 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking udftools (2.3-1) ... 2024-09-21T02:10:51.987 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpython3-dev:amd64 (3.10.6-1~22.04.1) ... 2024-09-21T02:10:52.113 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up exfatprogs (1.1.3-1ubuntu0.1) ... 2024-09-21T02:10:52.239 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up gawk (1:5.1.0-1ubuntu0.1) ... 2024-09-21T02:10:52.525 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up f2fs-tools (1.14.0-2build1) ... 2024-09-21T02:10:52.660 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-09-21T02:10:52.786 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-09-21T02:10:52.912 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up e2fsprogs (1.46.5-2ubuntu1.2) ... 2024-09-21T02:10:53.182 INFO:teuthology.orchestra.run.smithi096.stdout:update-initramfs: deferring update (trigger activated) 2024-09-21T02:10:53.731 INFO:teuthology.orchestra.run.smithi096.stdout:e2scrub_all.service is a disabled or a static unit not running, not starting it. 2024-09-21T02:10:53.834 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-09-21T02:10:53.969 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpmemblk1:amd64 (1.11.1-3build1) ... 2024-09-21T02:10:54.095 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up uuid-runtime (2.37.2-4ubuntu3.4) ... 2024-09-21T02:10:54.978 INFO:teuthology.orchestra.run.smithi096.stdout:uuidd.service is a disabled or a static unit not running, not starting it. 2024-09-21T02:10:55.038 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up udftools (2.3-1) ... 2024-09-21T02:10:55.181 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libdlm3:amd64 (4.1.1-1build1) ... 2024-09-21T02:10:55.349 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libgdbm-dev:amd64 (1.23-1) ... 2024-09-21T02:10:55.492 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-09-21T02:10:55.618 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-09-21T02:10:55.744 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liburing-dev:amd64 (2.1-2build1) ... 2024-09-21T02:10:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpython3-stdlib:amd64 (3.10.6-1~22.04.1) ... 2024-09-21T02:10:55.996 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-09-21T02:10:56.123 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3 (3.10.6-1~22.04.1) ... 2024-09-21T02:10:56.220 INFO:teuthology.orchestra.run.smithi096.stdout:running python rtupdate hooks for python3.10... 2024-09-21T02:10:56.446 INFO:teuthology.orchestra.run.smithi096.stdout:running python post-rtupdate hooks for python3.10... 2024-09-21T02:10:56.592 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up fio (3.28-1) ... 2024-09-21T02:10:57.168 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-venv (3.10.6-1~22.04.1) ... 2024-09-21T02:10:57.299 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-dev (3.10.6-1~22.04.1) ... 2024-09-21T02:10:57.423 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ocfs2-tools (1.8.7-1build1) ... 2024-09-21T02:10:57.696 INFO:teuthology.orchestra.run.smithi096.stdout:debconf: unable to initialize frontend: Dialog 2024-09-21T02:10:57.696 INFO:teuthology.orchestra.run.smithi096.stdout:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-09-21T02:10:57.696 INFO:teuthology.orchestra.run.smithi096.stdout:debconf: falling back to frontend: Readline 2024-09-21T02:10:58.475 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for initramfs-tools (0.140ubuntu13.1) ... 2024-09-21T02:10:58.572 INFO:teuthology.orchestra.run.smithi096.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T02:11:05.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:11:05.274 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:11:15.994 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:11:16.146 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:11:17.440 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:11:17.944 DEBUG:teuthology.orchestra.run.smithi096:> sudo useradd -m fsgqa 2024-09-21T02:11:17.955 INFO:teuthology.orchestra.run.smithi096.stderr:useradd: user 'fsgqa' already exists 2024-09-21T02:11:17.956 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-09-21T02:11:17.956 DEBUG:teuthology.orchestra.run.smithi096:> sudo groupadd fsgqa 2024-09-21T02:11:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:groupadd: group 'fsgqa' already exists 2024-09-21T02:11:18.009 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-09-21T02:11:18.010 DEBUG:teuthology.orchestra.run.smithi096:> sudo useradd fsgqa2 2024-09-21T02:11:18.435 DEBUG:teuthology.orchestra.run.smithi096:> sudo useradd 123456-fsgqa 2024-09-21T02:11:18.823 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-09-21T02:11:18.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 -- 172.21.15.96:0/1837094912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd460003a60 msgr2=0x7fd460003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/1837094912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd460003a60 0x7fd460003ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd4700016f0 tx=0x7fd47001c790 comp rx=0 tx=0).stop 2024-09-21T02:11:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 -- 172.21.15.96:0/1837094912 shutdown_connections 2024-09-21T02:11:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/1837094912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd460003a60 0x7fd460003ed0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 -- 172.21.15.96:0/1837094912 >> 172.21.15.96:0/1837094912 conn(0x7fd474073d10 msgr2=0x7fd474074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 -- 172.21.15.96:0/1837094912 shutdown_connections 2024-09-21T02:11:18.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.983+0000 7fd47ba2a640 1 -- 172.21.15.96:0/1837094912 wait complete. 2024-09-21T02:11:18.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 Processor -- start 2024-09-21T02:11:18.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 -- start start 2024-09-21T02:11:18.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd460003a60 0x7fd4740c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4741f9510 0x7fd474148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 0x7fd474152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd474152430 con 0x7fd4740c8df0 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741522b0 con 0x7fd4741f9510 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd4741525b0 con 0x7fd460003a60 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 0x7fd474152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 0x7fd474152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41282/0 (socket says 172.21.15.96:41282) 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 -- 172.21.15.96:0/705583653 learned_addr learned my addr 172.21.15.96:0/705583653 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd4797c6640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd460003a60 0x7fd4740c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd478fc5640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4741f9510 0x7fd474148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 -- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd460003a60 msgr2=0x7fd4740c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd460003a60 0x7fd4740c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:18.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 -- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4741f9510 msgr2=0x7fd474148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4741f9510 0x7fd474148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 -- 172.21.15.96:0/705583653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4740ca0f0 con 0x7fd4740c8df0 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd479fc7640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 0x7fd474152820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd474070620 tx=0x7fd46c00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003db0 con 0x7fd4740c8df0 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd46c018070 con 0x7fd4740c8df0 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4740c8310 con 0x7fd4740c8df0 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00cc10 con 0x7fd4740c8df0 2024-09-21T02:11:18.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.987+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd474153360 con 0x7fd4740c8df0 2024-09-21T02:11:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.991+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4740cc9f0 con 0x7fd4740c8df0 2024-09-21T02:11:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.991+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd46c004030 con 0x7fd4740c8df0 2024-09-21T02:11:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.991+0000 7fd469ffb640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd458073200 0x7fd458075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.991+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd46c0952b0 con 0x7fd4740c8df0 2024-09-21T02:11:18.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.991+0000 7fd4797c6640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd458073200 0x7fd458075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:18.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.991+0000 7fd4797c6640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd458073200 0x7fd458075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4700016c0 tx=0x7fd4700a8040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:18.995+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd46c060200 con 0x7fd4740c8df0 2024-09-21T02:11:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.235+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd4740b18d0 con 0x7fd4740c8df0 2024-09-21T02:11:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.235+0000 7fd469ffb640 1 -- 172.21.15.96:0/705583653 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7fd46c05fb00 con 0x7fd4740c8df0 2024-09-21T02:11:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:11:19.243 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","modified":"2024-09-21T02:08:03.526770Z","created":"2024-09-21T02:08:03.526770Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3301","nonce":0},{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T02:11:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd458073200 msgr2=0x7fd458075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd458073200 0x7fd458075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4700016c0 tx=0x7fd4700a8040 comp rx=0 tx=0).stop 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 msgr2=0x7fd474152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 0x7fd474152820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd474070620 tx=0x7fd46c00ee10 comp rx=0 tx=0).stop 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 shutdown_connections 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd458073200 0x7fd458075710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4740c8df0 0x7fd474152820 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4741f9510 0x7fd474148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 --2- 172.21.15.96:0/705583653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd460003a60 0x7fd4740c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 >> 172.21.15.96:0/705583653 conn(0x7fd474073d10 msgr2=0x7fd4740b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:19.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 shutdown_connections 2024-09-21T02:11:19.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.239+0000 7fd47ba2a640 1 -- 172.21.15.96:0/705583653 wait complete. 2024-09-21T02:11:19.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/1228375370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1815c5b0 msgr2=0x7f7e180cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/1228375370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1815c5b0 0x7f7e180cbaf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009840 tx=0x7f7e0801c820 comp rx=0 tx=0).stop 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/1228375370 shutdown_connections 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/1228375370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1815c5b0 0x7f7e180cbaf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/1228375370 >> 172.21.15.96:0/1228375370 conn(0x7f7e1806eea0 msgr2=0x7f7e1806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/1228375370 shutdown_connections 2024-09-21T02:11:19.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/1228375370 wait complete. 2024-09-21T02:11:19.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 Processor -- start 2024-09-21T02:11:19.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.419+0000 7f7e1cf4c640 1 -- start start 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 0x7f7e18156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1815c5b0 0x7f7e1814d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e181fa1a0 0x7f7e18158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18169a50 con 0x7f7e180ced00 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181698d0 con 0x7f7e181fa1a0 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7e18169bd0 con 0x7f7e1815c5b0 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 0x7f7e18156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 0x7f7e18156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41300/0 (socket says 172.21.15.96:41300) 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 -- 172.21.15.96:0/4205797742 learned_addr learned my addr 172.21.15.96:0/4205797742 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e16d9d640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e181fa1a0 0x7f7e18158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:19.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e15d9b640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1815c5b0 0x7f7e1814d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 -- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1815c5b0 msgr2=0x7f7e1814d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1815c5b0 0x7f7e1814d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 -- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e181fa1a0 msgr2=0x7f7e18158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e181fa1a0 0x7f7e18158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 -- 172.21.15.96:0/4205797742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e18156eb0 con 0x7f7e180ced00 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1659c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 0x7f7e18156540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000ba50 tx=0x7f7e1000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10019070 con 0x7f7e180ced00 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e1000cd10 con 0x7f7e180ced00 2024-09-21T02:11:19.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100146a0 con 0x7f7e180ced00 2024-09-21T02:11:19.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e18157190 con 0x7f7e180ced00 2024-09-21T02:11:19.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181f99a0 con 0x7f7e180ced00 2024-09-21T02:11:19.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e18075b80 con 0x7f7e180ced00 2024-09-21T02:11:19.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.423+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7e1001c070 con 0x7f7e180ced00 2024-09-21T02:11:19.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.427+0000 7f7e0effd640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7df8073200 0x7f7df8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:19.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.427+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7e10094070 con 0x7f7e180ced00 2024-09-21T02:11:19.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.427+0000 7f7e15d9b640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7df8073200 0x7f7df8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:19.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.427+0000 7f7e15d9b640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7df8073200 0x7f7df8075710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e180c3850 tx=0x7f7e00009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:19.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.431+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7e1005e920 con 0x7f7e180ced00 2024-09-21T02:11:19.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.663+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f7e180cbaf0 con 0x7f7e180ced00 2024-09-21T02:11:19.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.663+0000 7f7e0effd640 1 -- 172.21.15.96:0/4205797742 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v29) ==== 82+0+151 (secure 0 0 0) 0x7f7e10063350 con 0x7f7e180ced00 2024-09-21T02:11:19.669 INFO:teuthology.orchestra.run.smithi096.stdout:[client.admin] 2024-09-21T02:11:19.669 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCDKu5mskKQLBAAY/5fVutYOm2AszxhJIWVDA== 2024-09-21T02:11:19.669 INFO:teuthology.orchestra.run.smithi096.stdout: caps mds = "allow *" 2024-09-21T02:11:19.669 INFO:teuthology.orchestra.run.smithi096.stdout: caps mgr = "allow *" 2024-09-21T02:11:19.670 INFO:teuthology.orchestra.run.smithi096.stdout: caps mon = "allow *" 2024-09-21T02:11:19.670 INFO:teuthology.orchestra.run.smithi096.stdout: caps osd = "allow *" 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7df8073200 msgr2=0x7f7df8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7df8073200 0x7f7df8075710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e180c3850 tx=0x7f7e00009040 comp rx=0 tx=0).stop 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 msgr2=0x7f7e18156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 0x7f7e18156540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000ba50 tx=0x7f7e1000bf10 comp rx=0 tx=0).stop 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 shutdown_connections 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f7df8073200 0x7f7df8075710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e181fa1a0 0x7f7e18158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1815c5b0 0x7f7e1814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 --2- 172.21.15.96:0/4205797742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e180ced00 0x7f7e18156540 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:19.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 >> 172.21.15.96:0/4205797742 conn(0x7f7e1806eea0 msgr2=0x7f7e1806cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:19.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 shutdown_connections 2024-09-21T02:11:19.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:11:19.667+0000 7f7e1cf4c640 1 -- 172.21.15.96:0/4205797742 wait complete. 2024-09-21T02:11:19.692 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-21T02:11:19.696 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.VOmtMxMrck 2024-09-21T02:11:19.696 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:11:19.696 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/tmp/tmp.VOmtMxMrck 2024-09-21T02:11:19.743 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:11:19.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/tmp/tmp.trp2hAHaMUxfstests-dev/local.config 2024-09-21T02:11:19.798 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.cAh74ajS05test export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.i5J4KxFJ38scratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.96:6789 -k /tmp/tmp.VOmtMxMrck --client_mountpoint /test" 2024-09-21T02:11:19.799 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:11:19.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/tmp/tmp.trp2hAHaMUxfstests-dev/ceph.exclude 2024-09-21T02:11:19.854 DEBUG:teuthology.orchestra.run.smithi096:> (cd /tmp/tmp.trp2hAHaMUxfstests-dev && exec sudo make) 2024-09-21T02:11:19.915 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize -cfi 2024-09-21T02:11:20.203 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-21T02:11:20.203 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file './config.guess' 2024-09-21T02:11:20.207 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file './config.sub' 2024-09-21T02:11:20.213 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file './install-sh' 2024-09-21T02:11:20.219 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file './ltmain.sh' 2024-09-21T02:11:20.247 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-21T02:11:20.248 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-21T02:11:20.254 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-21T02:11:20.259 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-21T02:11:20.265 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-21T02:11:20.269 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-21T02:11:20.273 INFO:teuthology.orchestra.run.smithi096.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-21T02:11:20.274 INFO:teuthology.orchestra.run.smithi096.stdout:cp include/install-sh . 2024-09-21T02:11:20.275 INFO:teuthology.orchestra.run.smithi096.stdout:aclocal -I m4 2024-09-21T02:11:21.344 INFO:teuthology.orchestra.run.smithi096.stdout:autoheader 2024-09-21T02:11:21.714 INFO:teuthology.orchestra.run.smithi096.stdout:autoconf 2024-09-21T02:11:22.115 INFO:teuthology.orchestra.run.smithi096.stdout:./configure \ 2024-09-21T02:11:22.116 INFO:teuthology.orchestra.run.smithi096.stdout: --libexecdir=/usr/lib \ 2024-09-21T02:11:22.116 INFO:teuthology.orchestra.run.smithi096.stdout: --exec_prefix=/var/lib 2024-09-21T02:11:22.538 INFO:teuthology.orchestra.run.smithi096.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-21T02:11:22.538 INFO:teuthology.orchestra.run.smithi096.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-21T02:11:22.581 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to print strings... printf 2024-09-21T02:11:22.582 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gcc... gcc 2024-09-21T02:11:22.818 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether the C compiler works... yes 2024-09-21T02:11:22.818 INFO:teuthology.orchestra.run.smithi096.stdout:checking for C compiler default output file name... a.out 2024-09-21T02:11:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:checking for suffix of executables... 2024-09-21T02:11:22.935 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether we are cross compiling... no 2024-09-21T02:11:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:checking for suffix of object files... o 2024-09-21T02:11:22.974 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether the compiler supports GNU C... yes 2024-09-21T02:11:22.990 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether gcc accepts -g... yes 2024-09-21T02:11:23.142 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gcc option to enable C11 features... none needed 2024-09-21T02:11:23.147 INFO:teuthology.orchestra.run.smithi096.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-21T02:11:23.150 INFO:teuthology.orchestra.run.smithi096.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-21T02:11:23.152 INFO:teuthology.orchestra.run.smithi096.stdout:checking for egrep... /usr/bin/grep -E 2024-09-21T02:11:23.154 INFO:teuthology.orchestra.run.smithi096.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-21T02:11:23.159 INFO:teuthology.orchestra.run.smithi096.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-21T02:11:23.161 INFO:teuthology.orchestra.run.smithi096.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-21T02:11:23.184 INFO:teuthology.orchestra.run.smithi096.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-21T02:11:23.206 INFO:teuthology.orchestra.run.smithi096.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-21T02:11:23.206 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether ln -s works... yes 2024-09-21T02:11:23.219 INFO:teuthology.orchestra.run.smithi096.stdout:checking the maximum length of command line arguments... 1572864 2024-09-21T02:11:23.223 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-21T02:11:23.223 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-21T02:11:23.223 INFO:teuthology.orchestra.run.smithi096.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-21T02:11:23.223 INFO:teuthology.orchestra.run.smithi096.stdout:checking for objdump... objdump 2024-09-21T02:11:23.224 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to recognize dependent libraries... pass_all 2024-09-21T02:11:23.224 INFO:teuthology.orchestra.run.smithi096.stdout:checking for dlltool... no 2024-09-21T02:11:23.224 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-21T02:11:23.225 INFO:teuthology.orchestra.run.smithi096.stdout:checking for ar... ar 2024-09-21T02:11:23.264 INFO:teuthology.orchestra.run.smithi096.stdout:checking for archiver @FILE support... @ 2024-09-21T02:11:23.316 INFO:teuthology.orchestra.run.smithi096.stdout:checking for strip... strip 2024-09-21T02:11:23.317 INFO:teuthology.orchestra.run.smithi096.stdout:checking for ranlib... ranlib 2024-09-21T02:11:23.317 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gawk... gawk 2024-09-21T02:11:23.326 INFO:teuthology.orchestra.run.smithi096.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-21T02:11:23.328 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sysroot... no 2024-09-21T02:11:23.331 INFO:teuthology.orchestra.run.smithi096.stdout:checking for a working dd... /usr/bin/dd 2024-09-21T02:11:23.335 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-21T02:11:23.348 INFO:teuthology.orchestra.run.smithi096.stdout:checking for mt... mt 2024-09-21T02:11:23.370 INFO:teuthology.orchestra.run.smithi096.stdout:checking if mt is a manifest tool... no 2024-09-21T02:11:23.400 INFO:teuthology.orchestra.run.smithi096.stdout:checking for stdio.h... yes 2024-09-21T02:11:23.445 INFO:teuthology.orchestra.run.smithi096.stdout:checking for stdlib.h... yes 2024-09-21T02:11:23.474 INFO:teuthology.orchestra.run.smithi096.stdout:checking for string.h... yes 2024-09-21T02:11:23.502 INFO:teuthology.orchestra.run.smithi096.stdout:checking for inttypes.h... yes 2024-09-21T02:11:23.529 INFO:teuthology.orchestra.run.smithi096.stdout:checking for stdint.h... yes 2024-09-21T02:11:23.553 INFO:teuthology.orchestra.run.smithi096.stdout:checking for strings.h... yes 2024-09-21T02:11:23.579 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/stat.h... yes 2024-09-21T02:11:23.605 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/types.h... yes 2024-09-21T02:11:23.645 INFO:teuthology.orchestra.run.smithi096.stdout:checking for unistd.h... yes 2024-09-21T02:11:23.682 INFO:teuthology.orchestra.run.smithi096.stdout:checking for dlfcn.h... yes 2024-09-21T02:11:23.685 INFO:teuthology.orchestra.run.smithi096.stdout:checking for objdir... .libs 2024-09-21T02:11:23.767 INFO:teuthology.orchestra.run.smithi096.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-21T02:11:23.767 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-21T02:11:23.781 INFO:teuthology.orchestra.run.smithi096.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-21T02:11:24.086 INFO:teuthology.orchestra.run.smithi096.stdout:checking if gcc static flag -static works... yes 2024-09-21T02:11:24.115 INFO:teuthology.orchestra.run.smithi096.stdout:checking if gcc supports -c -o file.o... yes 2024-09-21T02:11:24.116 INFO:teuthology.orchestra.run.smithi096.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-21T02:11:24.125 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-21T02:11:24.153 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether -lc should be explicitly linked in... no 2024-09-21T02:11:24.225 INFO:teuthology.orchestra.run.smithi096.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-21T02:11:24.225 INFO:teuthology.orchestra.run.smithi096.stdout:checking how to hardcode library paths into programs... immediate 2024-09-21T02:11:24.233 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether stripping libraries is possible... yes 2024-09-21T02:11:24.233 INFO:teuthology.orchestra.run.smithi096.stdout:checking if libtool supports shared libraries... yes 2024-09-21T02:11:24.233 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether to build shared libraries... yes 2024-09-21T02:11:24.233 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether to build static libraries... yes 2024-09-21T02:11:24.242 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gcc... (cached) gcc 2024-09-21T02:11:24.267 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-09-21T02:11:24.267 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-21T02:11:24.267 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-09-21T02:11:24.268 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gmake... /usr/bin/gmake 2024-09-21T02:11:24.268 INFO:teuthology.orchestra.run.smithi096.stderr:./configure: line 12753: test: $(SHELL): binary operator expected 2024-09-21T02:11:24.268 INFO:teuthology.orchestra.run.smithi096.stdout:checking for tar... /bin/tar 2024-09-21T02:11:24.269 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gzip... /bin/gzip 2024-09-21T02:11:24.278 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether gcc -MM is supported... yes 2024-09-21T02:11:24.279 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sort... /bin/sort 2024-09-21T02:11:24.279 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether ln -s works... yes 2024-09-21T02:11:24.280 INFO:teuthology.orchestra.run.smithi096.stdout:checking for rpm... no 2024-09-21T02:11:24.323 INFO:teuthology.orchestra.run.smithi096.stdout:checking for assert.h... yes 2024-09-21T02:11:24.355 INFO:teuthology.orchestra.run.smithi096.stdout:checking for bstring.h... no 2024-09-21T02:11:24.388 INFO:teuthology.orchestra.run.smithi096.stdout:checking for libgen.h... yes 2024-09-21T02:11:24.455 INFO:teuthology.orchestra.run.smithi096.stdout:checking for dirent.h... yes 2024-09-21T02:11:24.535 INFO:teuthology.orchestra.run.smithi096.stdout:checking for errno.h... yes 2024-09-21T02:11:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:checking for malloc.h... yes 2024-09-21T02:11:24.608 INFO:teuthology.orchestra.run.smithi096.stdout:checking for uuid.h... no 2024-09-21T02:11:24.646 INFO:teuthology.orchestra.run.smithi096.stdout:checking for uuid/uuid.h... yes 2024-09-21T02:11:24.685 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/uuid.h... no 2024-09-21T02:11:24.720 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/file.h... yes 2024-09-21T02:11:24.753 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/fcntl.h... yes 2024-09-21T02:11:24.784 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/syssgi.h... no 2024-09-21T02:11:24.841 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/param.h... yes 2024-09-21T02:11:24.841 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/stat.h... (cached) yes 2024-09-21T02:11:24.872 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/statvfs.h... yes 2024-09-21T02:11:24.904 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/time.h... yes 2024-09-21T02:11:24.955 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/ioctl.h... yes 2024-09-21T02:11:24.990 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/wait.h... yes 2024-09-21T02:11:24.990 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/types.h... (cached) yes 2024-09-21T02:11:24.991 INFO:teuthology.orchestra.run.smithi096.stdout:checking for strings.h... (cached) yes 2024-09-21T02:11:25.021 INFO:teuthology.orchestra.run.smithi096.stdout:checking for err.h... yes 2024-09-21T02:11:25.053 INFO:teuthology.orchestra.run.smithi096.stdout:checking for linux/falloc.h... yes 2024-09-21T02:11:25.083 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-21T02:11:25.112 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-21T02:11:25.151 INFO:teuthology.orchestra.run.smithi096.stdout:checking for xfs/platform_defs.h... no 2024-09-21T02:11:25.180 INFO:teuthology.orchestra.run.smithi096.stdout:checking for btrfs/ioctl.h... no 2024-09-21T02:11:25.215 INFO:teuthology.orchestra.run.smithi096.stdout:checking for cifs/ioctl.h... no 2024-09-21T02:11:25.247 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/mman.h... yes 2024-09-21T02:11:25.278 INFO:teuthology.orchestra.run.smithi096.stdout:checking for linux/ext4.h... no 2024-09-21T02:11:25.290 INFO:teuthology.orchestra.run.smithi096.stdout:checking for xfs/xfs_log_format.h... no 2024-09-21T02:11:25.328 INFO:teuthology.orchestra.run.smithi096.stdout:checking for uuid_compare... no 2024-09-21T02:11:25.361 INFO:teuthology.orchestra.run.smithi096.stdout:checking for uuid_compare in -luuid... yes 2024-09-21T02:11:25.475 INFO:teuthology.orchestra.run.smithi096.stdout:checking for xfs/xfs.h... yes 2024-09-21T02:11:25.491 INFO:teuthology.orchestra.run.smithi096.stdout:checking for xfs/libxfs.h... no 2024-09-21T02:11:25.535 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gcc options needed to detect all undeclared functions... none needed 2024-09-21T02:11:25.556 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-21T02:11:25.601 INFO:teuthology.orchestra.run.smithi096.stdout:checking for xfs/xqm.h... yes 2024-09-21T02:11:25.661 INFO:teuthology.orchestra.run.smithi096.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-21T02:11:25.697 INFO:teuthology.orchestra.run.smithi096.stdout:checking for xfs/handle.h... yes 2024-09-21T02:11:25.731 INFO:teuthology.orchestra.run.smithi096.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-21T02:11:25.765 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/acl.h... yes 2024-09-21T02:11:25.806 INFO:teuthology.orchestra.run.smithi096.stdout:checking for acl/libacl.h... yes 2024-09-21T02:11:25.842 INFO:teuthology.orchestra.run.smithi096.stdout:checking for acl_init in -lacl... yes 2024-09-21T02:11:25.872 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gdbm-ndbm.h... no 2024-09-21T02:11:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:checking for gdbm/ndbm.h... no 2024-09-21T02:11:25.936 INFO:teuthology.orchestra.run.smithi096.stdout:checking for ndbm.h... no 2024-09-21T02:11:25.974 INFO:teuthology.orchestra.run.smithi096.stdout:checking for libaio.h... yes 2024-09-21T02:11:26.022 INFO:teuthology.orchestra.run.smithi096.stdout:checking for liburing.h... yes 2024-09-21T02:11:26.060 INFO:teuthology.orchestra.run.smithi096.stdout:checking for sys/capability.h... yes 2024-09-21T02:11:26.090 INFO:teuthology.orchestra.run.smithi096.stdout:checking for linux/fiemap.h... yes 2024-09-21T02:11:26.129 INFO:teuthology.orchestra.run.smithi096.stdout:checking for fallocate... yes 2024-09-21T02:11:26.165 INFO:teuthology.orchestra.run.smithi096.stdout:checking for open_by_handle_at... yes 2024-09-21T02:11:26.196 INFO:teuthology.orchestra.run.smithi096.stdout:checking for linux/fs.h... yes 2024-09-21T02:11:26.229 INFO:teuthology.orchestra.run.smithi096.stdout:checking for btrfsutil.h... no 2024-09-21T02:11:26.268 INFO:teuthology.orchestra.run.smithi096.stdout:checking for copy_file_range... yes 2024-09-21T02:11:26.308 INFO:teuthology.orchestra.run.smithi096.stdout:checking for SEEK_DATA... yes 2024-09-21T02:11:26.366 INFO:teuthology.orchestra.run.smithi096.stdout:checking for BMV_OF_SHARED... yes 2024-09-21T02:11:26.413 INFO:teuthology.orchestra.run.smithi096.stdout:checking for nftw... yes 2024-09-21T02:11:26.453 INFO:teuthology.orchestra.run.smithi096.stdout:checking for RLIMIT_NOFILE... yes 2024-09-21T02:11:26.492 INFO:teuthology.orchestra.run.smithi096.stdout:checking for new enough XFS_IOC_EXCHANGE_RANGE... no 2024-09-21T02:11:26.529 INFO:teuthology.orchestra.run.smithi096.stdout:checking for FICLONE... yes 2024-09-21T02:11:26.563 INFO:teuthology.orchestra.run.smithi096.stdout:checking for renameat2... yes 2024-09-21T02:11:26.597 INFO:teuthology.orchestra.run.smithi096.stdout:checking for reallocarray... yes 2024-09-21T02:11:26.626 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct mount_attr... yes 2024-09-21T02:11:26.670 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-21T02:11:26.700 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-21T02:11:26.733 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-21T02:11:26.764 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-21T02:11:26.796 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-21T02:11:26.828 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-21T02:11:26.862 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-21T02:11:26.895 INFO:teuthology.orchestra.run.smithi096.stdout:checking for linux/btrfs.h... yes 2024-09-21T02:11:26.927 INFO:teuthology.orchestra.run.smithi096.stdout:checking for linux/btrfs_tree.h... yes 2024-09-21T02:11:26.945 INFO:teuthology.orchestra.run.smithi096.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-21T02:11:26.977 INFO:teuthology.orchestra.run.smithi096.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-21T02:11:26.994 INFO:teuthology.orchestra.run.smithi096.stdout:configure: creating ./config.status 2024-09-21T02:11:27.393 INFO:teuthology.orchestra.run.smithi096.stdout:config.status: creating include/builddefs 2024-09-21T02:11:27.407 INFO:teuthology.orchestra.run.smithi096.stdout:config.status: creating include/config.h 2024-09-21T02:11:27.417 INFO:teuthology.orchestra.run.smithi096.stdout:config.status: executing libtool commands 2024-09-21T02:11:27.459 INFO:teuthology.orchestra.run.smithi096.stdout:Building include 2024-09-21T02:11:27.463 INFO:teuthology.orchestra.run.smithi096.stdout:Building lib 2024-09-21T02:11:27.465 INFO:teuthology.orchestra.run.smithi096.stdout: [LTDEP] 2024-09-21T02:11:27.606 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dataascii.lo 2024-09-21T02:11:27.716 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] databin.lo 2024-09-21T02:11:27.866 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] datapid.lo 2024-09-21T02:11:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] file_lock.lo 2024-09-21T02:11:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] forker.lo 2024-09-21T02:11:28.266 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] pattern.lo 2024-09-21T02:11:28.382 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] open_flags.lo 2024-09-21T02:11:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] random_range.lo 2024-09-21T02:11:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] string_to_tokens.lo 2024-09-21T02:11:28.881 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] str_to_bytes.lo 2024-09-21T02:11:29.033 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] tlibio.lo 2024-09-21T02:11:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] write_log.lo 2024-09-21T02:11:29.361 INFO:teuthology.orchestra.run.smithi096.stderr:write_log.c: In function 'wlog_record_write': 2024-09-21T02:11:29.361 INFO:teuthology.orchestra.run.smithi096.stderr:write_log.c:205:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:29.361 INFO:teuthology.orchestra.run.smithi096.stderr: 205 | write(wfile->w_afd, wbuf, reclen); 2024-09-21T02:11:29.361 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:29.361 INFO:teuthology.orchestra.run.smithi096.stderr:write_log.c:209:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:29.362 INFO:teuthology.orchestra.run.smithi096.stderr: 209 | write(wfile->w_rfd, wbuf, reclen); 2024-09-21T02:11:29.362 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] random.lo 2024-09-21T02:11:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: [LD] 2024-09-21T02:11:29.726 INFO:teuthology.orchestra.run.smithi096.stdout:Building ltp 2024-09-21T02:11:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: [DEP] 2024-09-21T02:11:29.897 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] doio 2024-09-21T02:11:30.605 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fsstress 2024-09-21T02:11:32.122 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fsx 2024-09-21T02:11:32.915 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] growfiles 2024-09-21T02:11:33.428 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] iogen 2024-09-21T02:11:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:iogen.c: In function 'main': 2024-09-21T02:11:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:iogen.c:280:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:33.552 INFO:teuthology.orchestra.run.smithi096.stderr: 280 | write(outfd, (char *)&req, sizeof(req)); 2024-09-21T02:11:33.552 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:iogen.c: In function 'parse_cmdline': 2024-09-21T02:11:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:iogen.c:1535:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:33.556 INFO:teuthology.orchestra.run.smithi096.stderr: 1535 | getcwd(fptr->f_path, 2024-09-21T02:11:33.556 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:33.556 INFO:teuthology.orchestra.run.smithi096.stderr: 1536 | sizeof(fptr->f_path)-1); 2024-09-21T02:11:33.556 INFO:teuthology.orchestra.run.smithi096.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-stress 2024-09-21T02:11:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:aio-stress.c: In function 'setup_shared_mem': 2024-09-21T02:11:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:aio-stress.c:1013:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:33.978 INFO:teuthology.orchestra.run.smithi096.stderr: 1013 | ftruncate(fd, total_ram); 2024-09-21T02:11:33.978 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:34.311 INFO:teuthology.orchestra.run.smithi096.stdout:Building src 2024-09-21T02:11:34.318 INFO:teuthology.orchestra.run.smithi096.stdout: [DEP] 2024-09-21T02:11:35.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:11:35.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:11:36.435 INFO:teuthology.orchestra.run.smithi096.stdout: [DEP] 2024-09-21T02:11:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: [DEP] 2024-09-21T02:11:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: [DEP] 2024-09-21T02:11:36.732 INFO:teuthology.orchestra.run.smithi096.stdout: [DEP] 2024-09-21T02:11:36.757 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dirstress 2024-09-21T02:11:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fill 2024-09-21T02:11:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fill2 2024-09-21T02:11:37.375 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] getpagesize 2024-09-21T02:11:37.508 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] holes 2024-09-21T02:11:37.701 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] lstat64 2024-09-21T02:11:37.883 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] nametest 2024-09-21T02:11:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:nametest.c: In function 'main': 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:nametest.c:236:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr: 236 | write(1, "\n", 1); 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:nametest.c:238:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr: 238 | write(1, ".", 1); 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:nametest.c:288:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr: 288 | write(1, ".", 1); 2024-09-21T02:11:38.009 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] permname 2024-09-21T02:11:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] randholes 2024-09-21T02:11:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] runas 2024-09-21T02:11:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] truncfile 2024-09-21T02:11:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] usemem 2024-09-21T02:11:39.091 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] mmapcat 2024-09-21T02:11:39.194 INFO:teuthology.orchestra.run.smithi096.stderr:mmapcat.c: In function 'main': 2024-09-21T02:11:39.194 INFO:teuthology.orchestra.run.smithi096.stderr:mmapcat.c:29:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.194 INFO:teuthology.orchestra.run.smithi096.stderr: 29 | write(1,ptr2,ptr - ptr2); 2024-09-21T02:11:39.194 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] append_reader 2024-09-21T02:11:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] append_writer 2024-09-21T02:11:39.515 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dirperf 2024-09-21T02:11:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] metaperf 2024-09-21T02:11:39.856 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c: In function 'crfiles': 2024-09-21T02:11:39.856 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c:166:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.856 INFO:teuthology.orchestra.run.smithi096.stderr: 166 | write(fd, buf, fsize); 2024-09-21T02:11:39.857 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:39.857 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c: In function 't_linkun': 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c:454:25: warning: ignoring return value of 'link' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr: 454 | link("a", *fnp); 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~ 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c: In function 't_chown': 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c:404:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr: 404 | chown(*fnp, 2, -1); 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c:406:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr: 406 | chown(*fnp, 1, -1); 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:39.858 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c: In function 'i_chown': 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c:291:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr: 291 | chown(*fnp, 1, -1); 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c: In function 'main': 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr:metaperf.c:152:9: warning: ignoring return value of 'chdir' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr: 152 | chdir(".."); 2024-09-21T02:11:39.859 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~ 2024-09-21T02:11:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] devzero 2024-09-21T02:11:40.208 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] feature 2024-09-21T02:11:40.447 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] alloc 2024-09-21T02:11:40.567 INFO:teuthology.orchestra.run.smithi096.stderr:alloc.c: In function 'main': 2024-09-21T02:11:40.567 INFO:teuthology.orchestra.run.smithi096.stderr:alloc.c:247:17: warning: ignoring return value of 'mkstemp' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:40.567 INFO:teuthology.orchestra.run.smithi096.stderr: 247 | mkstemp(tmpfile); 2024-09-21T02:11:40.567 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fault 2024-09-21T02:11:40.835 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fstest 2024-09-21T02:11:41.069 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_access_root 2024-09-21T02:11:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:t_access_root.c: In function 'main': 2024-09-21T02:11:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:t_access_root.c:55:5: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:41.169 INFO:teuthology.orchestra.run.smithi096.stderr: 55 | system(cmd); 2024-09-21T02:11:41.170 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~ 2024-09-21T02:11:41.209 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] godown 2024-09-21T02:11:41.367 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] resvtest 2024-09-21T02:11:41.545 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] writemod 2024-09-21T02:11:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] writev_on_pagefault 2024-09-21T02:11:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] makeextents 2024-09-21T02:11:42.039 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] itrash 2024-09-21T02:11:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] rename 2024-09-21T02:11:42.322 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] multi_open_unlink 2024-09-21T02:11:42.483 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] unwritten_sync 2024-09-21T02:11:42.656 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] genhashnames 2024-09-21T02:11:42.855 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_holes 2024-09-21T02:11:43.002 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_writev 2024-09-21T02:11:43.146 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_truncate_cmtime 2024-09-21T02:11:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dirhash_collide 2024-09-21T02:11:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_rename_overwrite 2024-09-21T02:11:43.592 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] holetest 2024-09-21T02:11:43.718 INFO:teuthology.orchestra.run.smithi096.stderr:holetest.c: In function 'pt_write_marker': 2024-09-21T02:11:43.718 INFO:teuthology.orchestra.run.smithi096.stderr:holetest.c:140:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:43.719 INFO:teuthology.orchestra.run.smithi096.stderr: 140 | pwrite(fd, &tid, sizeof(tid), i * page_size + pgoff); 2024-09-21T02:11:43.719 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:43.838 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_truncate_self 2024-09-21T02:11:43.979 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] af_unix 2024-09-21T02:11:44.131 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_stale_pmd 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_stale_pmd.c: In function 'main': 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_stale_pmd.c:42:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr: 42 | ftruncate(fd, 0); 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_stale_pmd.c:43:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr: 43 | ftruncate(fd, MiB(4)); 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_stale_pmd.c:55:9: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:44.247 INFO:teuthology.orchestra.run.smithi096.stderr: 55 | pwrite(fd, buffer, strlen(buffer), MiB(2)); 2024-09-21T02:11:44.247 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_cow_race 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_cow_race.c: In function 'main': 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_cow_race.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr: 61 | ftruncate(fd, 0); 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:t_mmap_cow_race.c:62:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr: 62 | ftruncate(fd, MiB(2)); 2024-09-21T02:11:44.387 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_fallocate 2024-09-21T02:11:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fsync-err 2024-09-21T02:11:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_write_ro 2024-09-21T02:11:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-21T02:11:45.044 INFO:teuthology.orchestra.run.smithi096.stderr:t_ext4_dax_journal_corruption.c: In function 'main': 2024-09-21T02:11:45.044 INFO:teuthology.orchestra.run.smithi096.stderr:t_ext4_dax_journal_corruption.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:45.044 INFO:teuthology.orchestra.run.smithi096.stderr: 61 | ftruncate(fd, 0); 2024-09-21T02:11:45.044 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-21T02:11:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_ofd_locks 2024-09-21T02:11:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_collision 2024-09-21T02:11:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] mmap-write-concurrent 2024-09-21T02:11:45.826 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_get_file_time 2024-09-21T02:11:45.971 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_create_short_dirs 2024-09-21T02:11:46.128 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_create_long_dirs 2024-09-21T02:11:46.279 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_enospc 2024-09-21T02:11:46.465 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_writev_overlap 2024-09-21T02:11:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] checkpoint_journal 2024-09-21T02:11:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] mmap-rw-fault 2024-09-21T02:11:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] allocstale 2024-09-21T02:11:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_cow_memory_failure 2024-09-21T02:11:47.307 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fake-dump-rootino 2024-09-21T02:11:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dio-buf-fault 2024-09-21T02:11:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] rewinddir-test 2024-09-21T02:11:47.756 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] readdir-while-renames 2024-09-21T02:11:47.911 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dio-append-buf-fault 2024-09-21T02:11:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] xfsctl 2024-09-21T02:11:48.260 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] bstat 2024-09-21T02:11:48.467 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mtab 2024-09-21T02:11:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:t_mtab.c: In function 'update_mtab': 2024-09-21T02:11:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:t_mtab.c:256:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:48.582 INFO:teuthology.orchestra.run.smithi096.stderr: 256 | chown (mounted_temp, sbuf.st_uid, sbuf.st_gid); 2024-09-21T02:11:48.582 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:48.662 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] getdevicesize 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] preallo_rw_pattern_reader 2024-09-21T02:11:48.911 INFO:teuthology.orchestra.run.smithi096.stderr:In file included from preallo_rw_pattern_reader.c:2: 2024-09-21T02:11:48.911 INFO:teuthology.orchestra.run.smithi096.stderr:iopat.c: In function 'main': 2024-09-21T02:11:48.911 INFO:teuthology.orchestra.run.smithi096.stderr:iopat.c:38:4: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:48.911 INFO:teuthology.orchestra.run.smithi096.stderr: 38 | read(fd, &x, 1048576); 2024-09-21T02:11:48.911 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] preallo_rw_pattern_writer 2024-09-21T02:11:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:In file included from preallo_rw_pattern_writer.c:2: 2024-09-21T02:11:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:iopat.c: In function 'main': 2024-09-21T02:11:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:iopat.c:34:4: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:49.063 INFO:teuthology.orchestra.run.smithi096.stderr: 34 | write(fd, &x, 1048576); 2024-09-21T02:11:49.063 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:49.098 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] ftrunc 2024-09-21T02:11:49.246 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] trunc 2024-09-21T02:11:49.404 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fs_perms 2024-09-21T02:11:49.509 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c: In function 'main': 2024-09-21T02:11:49.509 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c:54:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:49.509 INFO:teuthology.orchestra.run.smithi096.stderr: 54 | system("rm -f test.file"); 2024-09-21T02:11:49.509 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:49.509 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c: In function 'testsetup': 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c:67:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: 67 | system("cp testx.file test.file"); 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c:69:4: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: 69 | chown("test.file",cuserId,cgroupId); 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c: In function 'testfperm': 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c:123:5: warning: ignoring return value of 'seteuid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: 123 | seteuid(0); 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~ 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:fs_perms.c:124:5: warning: ignoring return value of 'setegid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: 124 | setegid(0); 2024-09-21T02:11:49.510 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~ 2024-09-21T02:11:49.557 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] testx 2024-09-21T02:11:49.667 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] looptest 2024-09-21T02:11:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] locktest 2024-09-21T02:11:50.199 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] unwritten_mmap 2024-09-21T02:11:50.351 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] bulkstat_unlink_test 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test.c: In function 'main': 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test.c:106:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr: 106 | write(fd[i], fname, sizeof(fname)); 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test.c:178:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr: 178 | system(fname); 2024-09-21T02:11:50.464 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~ 2024-09-21T02:11:50.549 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] deduperace 2024-09-21T02:11:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] bulkstat_unlink_test_modified 2024-09-21T02:11:50.863 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test_modified.c: In function 'main': 2024-09-21T02:11:50.863 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test_modified.c:83:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:50.863 INFO:teuthology.orchestra.run.smithi096.stderr: 83 | write(fd, fname, sizeof(fname)); 2024-09-21T02:11:50.863 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:50.863 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test_modified.c:139:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:50.863 INFO:teuthology.orchestra.run.smithi096.stderr: 139 | write(fd, fname, sizeof(fname)); 2024-09-21T02:11:50.864 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:50.864 INFO:teuthology.orchestra.run.smithi096.stderr:bulkstat_unlink_test_modified.c:185:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:50.864 INFO:teuthology.orchestra.run.smithi096.stderr: 185 | system(fname); 2024-09-21T02:11:50.864 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~ 2024-09-21T02:11:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_dir_offset 2024-09-21T02:11:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_futimens 2024-09-21T02:11:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_immutable 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c: In function 'create_test_area': 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2061:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr: 2061 | asprintf(&path, "%s/immutable.d", dir); 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2096:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr: 2096 | asprintf(&path, "%s/append-only.d", dir); 2024-09-21T02:11:51.335 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c: In function 'remove_test_area': 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2152:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr: 2152 | asprintf(&path, "%s/immutable.d", dir); 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2163:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr: 2163 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-09-21T02:11:51.336 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.337 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2175:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.337 INFO:teuthology.orchestra.run.smithi096.stderr: 2175 | asprintf(&path, "%s/append-only.d", dir); 2024-09-21T02:11:51.337 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.337 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2186:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: 2186 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2197:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: 2197 | asprintf(&path, "%s/append-only.f", dir); 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:2209:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: 2209 | asprintf(&path, "%s/immutable.f", dir); 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c: In function 'check_test_area': 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1788:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: 1788 | asprintf(&path, "%s/", dir); 2024-09-21T02:11:51.338 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.339 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1800:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.339 INFO:teuthology.orchestra.run.smithi096.stderr: 1800 | asprintf(&path, "%s/immutable.f", dir); 2024-09-21T02:11:51.339 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.339 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1812:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.339 INFO:teuthology.orchestra.run.smithi096.stderr: 1812 | asprintf(&path, "%s/append-only.f", dir); 2024-09-21T02:11:51.339 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.341 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1824:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.341 INFO:teuthology.orchestra.run.smithi096.stderr: 1824 | asprintf(&path, "%s/immutable.d", dir); 2024-09-21T02:11:51.341 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1837:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: 1837 | asprintf(&path, "%s/append-only.d", dir); 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1850:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: 1850 | asprintf(&path, "%s/immutable.d/file", dir); 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1862:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: 1862 | asprintf(&path, "%s/append-only.d/file", dir); 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1874:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: 1874 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-09-21T02:11:51.342 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1887:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: 1887 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c: In function 'test_immutable': 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:151:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: 151 | asprintf(&path, "%s/immutable.f", dir); 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:381:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: 381 | asprintf(&linkpath, "%s/immutable.f.hardlink", dir); 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:408:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: 408 | chown(path, 0, 0); 2024-09-21T02:11:51.343 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:485:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: 485 | asprintf(&linkpath, "%s/immutable.f.newname", dir); 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:532:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: 532 | asprintf(&path, "%s/immutable.d/file", dir); 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:553:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: 553 | chown(path, 0, 0); 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:556:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: 556 | asprintf(&linkpath, "%s/immutable.d/file.link", dir); 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:575:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: 575 | asprintf(&linkpath, "%s/immutable.d/file.newname", dir); 2024-09-21T02:11:51.344 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:595:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: 595 | asprintf(&path, "%s/immutable.d/newfile", dir); 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:619:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: 619 | asprintf(&path, "%s/immutable.d/newdir", dir); 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:631:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: 631 | asprintf(&path, "%s/immutable.d/dir/newfile-%d", dir, getuid()); 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:653:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: 653 | chown(path, 0, 0); 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:657:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.345 INFO:teuthology.orchestra.run.smithi096.stderr: 657 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:668:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: 668 | asprintf(&path, "%s/immutable.d", dir); 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:705:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: 705 | chown(path, 0, 0); 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:782:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: 782 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-09-21T02:11:51.346 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.348 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c: In function 'test_append': 2024-09-21T02:11:51.348 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:822:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.348 INFO:teuthology.orchestra.run.smithi096.stderr: 822 | asprintf(&path, "%s/append-only.f", dir); 2024-09-21T02:11:51.348 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1024:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr: 1024 | asprintf(&linkpath, "%s/append-only.f.hardlink", dir); 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1051:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr: 1051 | chown(path, 0, 0); 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1126:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr: 1126 | asprintf(&linkpath, "%s/append-only.f.newname", dir); 2024-09-21T02:11:51.349 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1495:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: 1495 | asprintf(&path, "%s/append-only.d/file", dir); 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1516:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: 1516 | chown(path, 0, 0); 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1519:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: 1519 | asprintf(&linkpath, "%s/append-only.d/file.link-%d", dir, getuid()); 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1529:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: 1529 | asprintf(&linkpath, "%s/append-only.d/file.symlink-%d", dir, getuid()); 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1539:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: 1539 | asprintf(&linkpath, "%s/append-only.d/file.newname", dir); 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1559:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: 1559 | asprintf(&path, "%s/append-only.d/newfile-%d", dir, getuid()); 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1573:11: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: 1573 | asprintf(&path, "%s/append-only.d/newdev-%d", dir, getuid()); 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1586:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: 1586 | asprintf(&path, "%s/append-only.d/newdir-%d", dir, getuid()); 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1596:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: 1596 | asprintf(&path, "%s/append-only.d/newdir-%d/newfile-%d", dir, getuid(), getuid()); 2024-09-21T02:11:51.351 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1618:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: 1618 | chown(path, 0, 0); 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1622:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: 1622 | asprintf(&path, "%s/append-only.d/dir/newfile-%d", dir, getuid()); 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1644:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: 1644 | chown(path, 0, 0); 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1648:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: 1648 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1659:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.352 INFO:teuthology.orchestra.run.smithi096.stderr: 1659 | asprintf(&path, "%s/append-only.d", dir); 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1692:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr: 1692 | chown(path, 0, 0); 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr:t_immutable.c:1769:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr: 1769 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-09-21T02:11:51.353 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] stale_handle 2024-09-21T02:11:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:stale_handle.c: In function 'main': 2024-09-21T02:11:52.096 INFO:teuthology.orchestra.run.smithi096.stderr:stale_handle.c:102:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.096 INFO:teuthology.orchestra.run.smithi096.stderr: 102 | system("echo 3 > /proc/sys/vm/drop_caches"); 2024-09-21T02:11:52.096 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.146 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] pwrite_mmap_blocked 2024-09-21T02:11:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_dir_offset2 2024-09-21T02:11:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] seek_sanity_test 2024-09-21T02:11:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c: In function 'test18': 2024-09-21T02:11:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:460:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.608 INFO:teuthology.orchestra.run.smithi096.stderr: 460 | ftruncate(fd, 0); 2024-09-21T02:11:52.608 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c: In function 'test_basic_support': 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:1207:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr: 1207 | ftruncate(fd, 0); 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:1256:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr: 1256 | ftruncate(fd, 0); 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c: In function 'get_io_sizes': 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:88:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.611 INFO:teuthology.orchestra.run.smithi096.stderr: 88 | ftruncate(fd, 0); 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:89:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: 89 | pwrite(fd, "a", 1, offset); 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:98:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: 98 | ftruncate(fd, 0); 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr:seek_sanity_test.c:99:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: 99 | pwrite(fd, "a", 1, offset); 2024-09-21T02:11:52.612 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:52.927 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] seek_copy_test 2024-09-21T02:11:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_readdir_1 2024-09-21T02:11:53.195 INFO:teuthology.orchestra.run.smithi096.stderr:t_readdir_1.c: In function 'main': 2024-09-21T02:11:53.195 INFO:teuthology.orchestra.run.smithi096.stderr:t_readdir_1.c:30:25: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:53.195 INFO:teuthology.orchestra.run.smithi096.stderr: 30 | read(fd, buf, 100); 2024-09-21T02:11:53.195 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_readdir_2 2024-09-21T02:11:53.367 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fsync-tester 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr:fsync-tester.c: In function 'main': 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr:fsync-tester.c:391:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr: 391 | read(fd, &seed, sizeof(seed)); 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr:fsync-tester.c:541:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr: 541 | system("reboot -fn"); 2024-09-21T02:11:53.465 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:53.466 INFO:teuthology.orchestra.run.smithi096.stderr:fsync-tester.c: In function 'drop_all_caches': 2024-09-21T02:11:53.466 INFO:teuthology.orchestra.run.smithi096.stderr:fsync-tester.c:90:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:53.466 INFO:teuthology.orchestra.run.smithi096.stderr: 90 | write(fd, value, sizeof(value)-1); 2024-09-21T02:11:53.466 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] nsexec 2024-09-21T02:11:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] cloner 2024-09-21T02:11:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] renameat2 2024-09-21T02:11:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_getcwd 2024-09-21T02:11:54.220 INFO:teuthology.orchestra.run.smithi096.stderr:t_getcwd.c: In function 'test_getcwd': 2024-09-21T02:11:54.221 INFO:teuthology.orchestra.run.smithi096.stderr:t_getcwd.c:21:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:54.221 INFO:teuthology.orchestra.run.smithi096.stderr: 21 | getcwd(cur_cwd, BUF_SIZE); 2024-09-21T02:11:54.221 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] e4compact 2024-09-21T02:11:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] test-nextquota 2024-09-21T02:11:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] punch-alternating 2024-09-21T02:11:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-21T02:11:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] listxattr 2024-09-21T02:11:55.140 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dio-interleaved 2024-09-21T02:11:55.316 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_dir_type 2024-09-21T02:11:55.474 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] dio-invalidate-cache 2024-09-21T02:11:55.574 INFO:teuthology.orchestra.run.smithi096.stderr:dio-invalidate-cache.c: In function 'main': 2024-09-21T02:11:55.574 INFO:teuthology.orchestra.run.smithi096.stderr:dio-invalidate-cache.c:286:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:55.575 INFO:teuthology.orchestra.run.smithi096.stderr: 286 | write(fd, buf, blksz * nr_child); 2024-09-21T02:11:55.575 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] stat_test 2024-09-21T02:11:55.981 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_encrypted_d_revalidate 2024-09-21T02:11:56.145 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] attr_replace_test 2024-09-21T02:11:56.294 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] swapon 2024-09-21T02:11:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] mkswap 2024-09-21T02:11:56.598 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_attr_corruption 2024-09-21T02:11:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_open_tmpfiles 2024-09-21T02:11:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fscrypt-crypt-util 2024-09-21T02:11:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] bulkstat_null_ocount 2024-09-21T02:11:57.737 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] splice-test 2024-09-21T02:11:57.914 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] chprojid_fail 2024-09-21T02:11:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] detached_mounts_propagation 2024-09-21T02:11:58.236 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] ext4_resize 2024-09-21T02:11:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_readdir_3 2024-09-21T02:11:58.561 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] splice2pipe 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:splice2pipe.c: In function 'prepare_pipe': 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:splice2pipe.c:57:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr: 57 | write(p[1], buffer, n); 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:splice2pipe.c:65:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr: 65 | read(p[0], buffer, n); 2024-09-21T02:11:58.669 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:11:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] uuid_ioctl 2024-09-21T02:11:58.884 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-21T02:11:59.043 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fiemap-fault 2024-09-21T02:11:59.186 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] min_dio_alignment 2024-09-21T02:11:59.323 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fiemap-tester 2024-09-21T02:11:59.591 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_stripealign 2024-09-21T02:11:59.742 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] open_by_handle 2024-09-21T02:11:59.999 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_mmap_dio 2024-09-21T02:12:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] uring_read_fault 2024-09-21T02:12:00.335 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] xfsfind 2024-09-21T02:12:00.535 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] t_reflink_read_race 2024-09-21T02:12:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] fssum 2024-09-21T02:12:01.200 INFO:teuthology.orchestra.run.smithi096.stdout:Building log-writes 2024-09-21T02:12:01.207 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] replay-log 2024-09-21T02:12:01.550 INFO:teuthology.orchestra.run.smithi096.stdout:Building perf 2024-09-21T02:12:01.554 INFO:teuthology.orchestra.run.smithi096.stdout:Building aio-dio-regress 2024-09-21T02:12:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aiocp 2024-09-21T02:12:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-21T02:12:01.973 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-append-write-read-race 2024-09-21T02:12:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-cow-race 2024-09-21T02:12:02.306 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-cycle-write 2024-09-21T02:12:02.406 INFO:teuthology.orchestra.run.smithi096.stderr:aio-dio-cycle-write.c: In function 'init_test': 2024-09-21T02:12:02.406 INFO:teuthology.orchestra.run.smithi096.stderr:aio-dio-cycle-write.c:83:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:12:02.406 INFO:teuthology.orchestra.run.smithi096.stderr: 83 | ftruncate(fd, buf_size); 2024-09-21T02:12:02.406 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:12:02.494 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-eof-race 2024-09-21T02:12:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-extend-stat 2024-09-21T02:12:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-fcntl-race 2024-09-21T02:12:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-hole-filling-race 2024-09-21T02:12:03.175 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-invalidate-failure 2024-09-21T02:12:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-invalidate-readahead 2024-09-21T02:12:03.514 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aiodio_sparse2 2024-09-21T02:12:03.627 INFO:teuthology.orchestra.run.smithi096.stderr:aiodio_sparse2.c: In function 'aiodio_sparse': 2024-09-21T02:12:03.627 INFO:teuthology.orchestra.run.smithi096.stderr:aiodio_sparse2.c:111:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:12:03.627 INFO:teuthology.orchestra.run.smithi096.stderr: 111 | ftruncate(fd, filesize); 2024-09-21T02:12:03.627 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-09-21T02:12:03.628 INFO:teuthology.orchestra.run.smithi096.stderr:aiodio_sparse2.c: In function 'dirty_freeblocks': 2024-09-21T02:12:03.628 INFO:teuthology.orchestra.run.smithi096.stderr:aiodio_sparse2.c:274:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-09-21T02:12:03.628 INFO:teuthology.orchestra.run.smithi096.stderr: 274 | ftruncate(fd, size); 2024-09-21T02:12:03.628 INFO:teuthology.orchestra.run.smithi096.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-21T02:12:03.753 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-subblock-eof-read 2024-09-21T02:12:03.904 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-dio-write-verify 2024-09-21T02:12:04.124 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-21T02:12:04.253 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-21T02:12:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] aio-last-ref-held-by-io 2024-09-21T02:12:04.552 INFO:teuthology.orchestra.run.smithi096.stdout:Building vfs 2024-09-21T02:12:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] vfstest 2024-09-21T02:12:05.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:05.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: [CC] mount-idmapped 2024-09-21T02:12:09.139 INFO:teuthology.orchestra.run.smithi096.stdout:Building m4 2024-09-21T02:12:09.144 INFO:teuthology.orchestra.run.smithi096.stdout:Building common 2024-09-21T02:12:09.148 INFO:teuthology.orchestra.run.smithi096.stdout:Building tests 2024-09-21T02:12:09.154 INFO:teuthology.orchestra.run.smithi096.stdout:Building btrfs 2024-09-21T02:12:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/btrfs/group.list 2024-09-21T02:12:09.228 INFO:teuthology.orchestra.run.smithi096.stdout:Building ceph 2024-09-21T02:12:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ceph/group.list 2024-09-21T02:12:09.250 INFO:teuthology.orchestra.run.smithi096.stdout:Building cifs 2024-09-21T02:12:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/cifs/group.list 2024-09-21T02:12:09.271 INFO:teuthology.orchestra.run.smithi096.stdout:Building ext4 2024-09-21T02:12:09.277 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ext4/group.list 2024-09-21T02:12:09.319 INFO:teuthology.orchestra.run.smithi096.stdout:Building f2fs 2024-09-21T02:12:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/f2fs/group.list 2024-09-21T02:12:09.345 INFO:teuthology.orchestra.run.smithi096.stdout:Building generic 2024-09-21T02:12:09.352 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/generic/group.list 2024-09-21T02:12:09.464 INFO:teuthology.orchestra.run.smithi096.stdout:Building nfs 2024-09-21T02:12:09.470 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/nfs/group.list 2024-09-21T02:12:09.487 INFO:teuthology.orchestra.run.smithi096.stdout:Building ocfs2 2024-09-21T02:12:09.493 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ocfs2/group.list 2024-09-21T02:12:09.507 INFO:teuthology.orchestra.run.smithi096.stdout:Building overlay 2024-09-21T02:12:09.513 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/overlay/group.list 2024-09-21T02:12:09.557 INFO:teuthology.orchestra.run.smithi096.stdout:Building perf 2024-09-21T02:12:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/perf/group.list 2024-09-21T02:12:09.575 INFO:teuthology.orchestra.run.smithi096.stdout:Building selftest 2024-09-21T02:12:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/selftest/group.list 2024-09-21T02:12:09.594 INFO:teuthology.orchestra.run.smithi096.stdout:Building tmpfs 2024-09-21T02:12:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/tmpfs/group.list 2024-09-21T02:12:09.615 INFO:teuthology.orchestra.run.smithi096.stdout:Building udf 2024-09-21T02:12:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/udf/group.list 2024-09-21T02:12:09.633 INFO:teuthology.orchestra.run.smithi096.stdout:Building xfs 2024-09-21T02:12:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/xfs/group.list 2024-09-21T02:12:09.755 DEBUG:teuthology.orchestra.run.smithi096:> (cd /tmp/tmp.trp2hAHaMUxfstests-dev && exec sudo make install) 2024-09-21T02:12:09.780 INFO:teuthology.orchestra.run.smithi096.stdout:Building include 2024-09-21T02:12:09.784 INFO:teuthology.orchestra.run.smithi096.stdout:Building lib 2024-09-21T02:12:09.789 INFO:teuthology.orchestra.run.smithi096.stdout:Building ltp 2024-09-21T02:12:09.794 INFO:teuthology.orchestra.run.smithi096.stdout:Building src 2024-09-21T02:12:09.832 INFO:teuthology.orchestra.run.smithi096.stdout:Building log-writes 2024-09-21T02:12:09.836 INFO:teuthology.orchestra.run.smithi096.stdout:Building perf 2024-09-21T02:12:09.839 INFO:teuthology.orchestra.run.smithi096.stdout:Building aio-dio-regress 2024-09-21T02:12:09.844 INFO:teuthology.orchestra.run.smithi096.stdout:Building vfs 2024-09-21T02:12:09.848 INFO:teuthology.orchestra.run.smithi096.stdout:Building m4 2024-09-21T02:12:09.852 INFO:teuthology.orchestra.run.smithi096.stdout:Building common 2024-09-21T02:12:09.855 INFO:teuthology.orchestra.run.smithi096.stdout:Building tests 2024-09-21T02:12:09.861 INFO:teuthology.orchestra.run.smithi096.stdout:Building btrfs 2024-09-21T02:12:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/btrfs/group.list 2024-09-21T02:12:09.935 INFO:teuthology.orchestra.run.smithi096.stdout:Building ceph 2024-09-21T02:12:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ceph/group.list 2024-09-21T02:12:09.958 INFO:teuthology.orchestra.run.smithi096.stdout:Building cifs 2024-09-21T02:12:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/cifs/group.list 2024-09-21T02:12:09.978 INFO:teuthology.orchestra.run.smithi096.stdout:Building ext4 2024-09-21T02:12:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ext4/group.list 2024-09-21T02:12:10.023 INFO:teuthology.orchestra.run.smithi096.stdout:Building f2fs 2024-09-21T02:12:10.029 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/f2fs/group.list 2024-09-21T02:12:10.048 INFO:teuthology.orchestra.run.smithi096.stdout:Building generic 2024-09-21T02:12:10.056 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/generic/group.list 2024-09-21T02:12:10.168 INFO:teuthology.orchestra.run.smithi096.stdout:Building nfs 2024-09-21T02:12:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/nfs/group.list 2024-09-21T02:12:10.192 INFO:teuthology.orchestra.run.smithi096.stdout:Building ocfs2 2024-09-21T02:12:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ocfs2/group.list 2024-09-21T02:12:10.215 INFO:teuthology.orchestra.run.smithi096.stdout:Building overlay 2024-09-21T02:12:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/overlay/group.list 2024-09-21T02:12:10.274 INFO:teuthology.orchestra.run.smithi096.stdout:Building perf 2024-09-21T02:12:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/perf/group.list 2024-09-21T02:12:10.295 INFO:teuthology.orchestra.run.smithi096.stdout:Building selftest 2024-09-21T02:12:10.301 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/selftest/group.list 2024-09-21T02:12:10.317 INFO:teuthology.orchestra.run.smithi096.stdout:Building tmpfs 2024-09-21T02:12:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/tmpfs/group.list 2024-09-21T02:12:10.336 INFO:teuthology.orchestra.run.smithi096.stdout:Building udf 2024-09-21T02:12:10.341 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/udf/group.list 2024-09-21T02:12:10.355 INFO:teuthology.orchestra.run.smithi096.stdout:Building xfs 2024-09-21T02:12:10.362 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/xfs/group.list 2024-09-21T02:12:10.467 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-09-21T02:12:10.470 INFO:teuthology.orchestra.run.smithi096.stdout:gmake[1]: Nothing to be done for 'install'. 2024-09-21T02:12:10.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-09-21T02:12:10.473 INFO:teuthology.orchestra.run.smithi096.stdout:gmake[1]: Nothing to be done for 'install'. 2024-09-21T02:12:10.473 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-09-21T02:12:10.476 INFO:teuthology.orchestra.run.smithi096.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-09-21T02:12:10.486 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-09-21T02:12:10.573 INFO:teuthology.orchestra.run.smithi096.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-09-21T02:12:10.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-09-21T02:12:10.605 INFO:teuthology.orchestra.run.smithi096.stdout:Building log-writes 2024-09-21T02:12:10.609 INFO:teuthology.orchestra.run.smithi096.stdout:Building perf 2024-09-21T02:12:10.612 INFO:teuthology.orchestra.run.smithi096.stdout:Building aio-dio-regress 2024-09-21T02:12:10.617 INFO:teuthology.orchestra.run.smithi096.stdout:Building vfs 2024-09-21T02:12:10.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake -C log-writes install 2024-09-21T02:12:10.622 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-09-21T02:12:10.631 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-09-21T02:12:10.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake -C perf install 2024-09-21T02:12:10.643 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-09-21T02:12:10.652 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-09-21T02:12:10.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-09-21T02:12:10.677 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-09-21T02:12:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-09-21T02:12:10.742 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake -C vfs install 2024-09-21T02:12:10.745 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-09-21T02:12:10.756 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-09-21T02:12:10.771 INFO:teuthology.orchestra.run.smithi096.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-09-21T02:12:10.781 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames dio-append-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment fiemap-tester t_stripealign open_by_handle t_mmap_dio uring_read_fault xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-09-21T02:12:12.422 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-09-21T02:12:12.558 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-09-21T02:12:12.592 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-09-21T02:12:12.594 INFO:teuthology.orchestra.run.smithi096.stdout:gmake[1]: Nothing to be done for 'install'. 2024-09-21T02:12:12.595 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-09-21T02:12:12.597 INFO:teuthology.orchestra.run.smithi096.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-09-21T02:12:12.608 INFO:teuthology.orchestra.run.smithi096.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-09-21T02:12:12.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-09-21T02:12:12.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/btrfs/ install 2024-09-21T02:12:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/btrfs/group.list 2024-09-21T02:12:12.843 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-09-21T02:12:12.853 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 330 331 332 /var/lib/xfstests/tests/btrfs 2024-09-21T02:12:13.905 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-09-21T02:12:13.917 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 330.out 331.out 332.out /var/lib/xfstests/tests/btrfs 2024-09-21T02:12:14.993 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ceph/ install 2024-09-21T02:12:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ceph/group.list 2024-09-21T02:12:15.016 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-09-21T02:12:15.028 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-09-21T02:12:15.053 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-09-21T02:12:15.065 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-09-21T02:12:15.090 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/cifs/ install 2024-09-21T02:12:15.093 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/cifs/group.list 2024-09-21T02:12:15.109 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-09-21T02:12:15.121 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-09-21T02:12:15.133 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-09-21T02:12:15.145 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-09-21T02:12:15.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ext4/ install 2024-09-21T02:12:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ext4/group.list 2024-09-21T02:12:15.209 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-09-21T02:12:15.221 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-09-21T02:12:15.437 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-09-21T02:12:15.451 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-09-21T02:12:15.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/f2fs/ install 2024-09-21T02:12:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/f2fs/group.list 2024-09-21T02:12:15.717 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-09-21T02:12:15.728 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 /var/lib/xfstests/tests/f2fs 2024-09-21T02:12:15.750 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-09-21T02:12:15.761 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out /var/lib/xfstests/tests/f2fs 2024-09-21T02:12:15.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/generic/ install 2024-09-21T02:12:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/generic/group.list 2024-09-21T02:12:15.917 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-09-21T02:12:15.930 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 /var/lib/xfstests/tests/generic 2024-09-21T02:12:18.386 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-09-21T02:12:18.399 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out /var/lib/xfstests/tests/generic 2024-09-21T02:12:20.869 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/nfs/ install 2024-09-21T02:12:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/nfs/group.list 2024-09-21T02:12:20.890 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-09-21T02:12:20.902 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-09-21T02:12:20.914 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-09-21T02:12:20.925 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-09-21T02:12:20.938 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ocfs2/ install 2024-09-21T02:12:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/ocfs2/group.list 2024-09-21T02:12:20.959 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-09-21T02:12:20.971 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-09-21T02:12:20.983 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-09-21T02:12:20.995 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-09-21T02:12:21.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/overlay/ install 2024-09-21T02:12:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/overlay/group.list 2024-09-21T02:12:21.062 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-09-21T02:12:21.074 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-09-21T02:12:21.407 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-09-21T02:12:21.418 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-09-21T02:12:21.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/perf/ install 2024-09-21T02:12:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/perf/group.list 2024-09-21T02:12:21.784 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-09-21T02:12:21.796 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-09-21T02:12:21.808 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-09-21T02:12:21.820 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-09-21T02:12:21.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/selftest/ install 2024-09-21T02:12:21.836 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/selftest/group.list 2024-09-21T02:12:21.852 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-09-21T02:12:21.864 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-09-21T02:12:21.890 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-09-21T02:12:21.901 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-09-21T02:12:21.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/tmpfs/ install 2024-09-21T02:12:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/tmpfs/group.list 2024-09-21T02:12:21.949 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-09-21T02:12:21.960 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-09-21T02:12:21.971 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-09-21T02:12:21.982 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-09-21T02:12:21.994 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/udf/ install 2024-09-21T02:12:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/udf/group.list 2024-09-21T02:12:22.011 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-09-21T02:12:22.022 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-09-21T02:12:22.035 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-09-21T02:12:22.046 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-09-21T02:12:22.059 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.trp2hAHaMUxfstests-dev/tests/xfs/ install 2024-09-21T02:12:22.064 INFO:teuthology.orchestra.run.smithi096.stdout: [GROUP] /tmp/tmp.trp2hAHaMUxfstests-dev/tests/xfs/group.list 2024-09-21T02:12:22.185 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-09-21T02:12:22.197 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 /var/lib/xfstests/tests/xfs 2024-09-21T02:12:24.607 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-09-21T02:12:24.619 INFO:teuthology.orchestra.run.smithi096.stdout:../../install-sh -o root -g root -m 644 021.out.default 021.out.parent 033.out.crc 033.out.default 071.out.32 071.out.64 216.out.64mblog 216.out.classic 614.out.lba1024 614.out.lba2048 614.out.lba4096 614.out.lba512 021.cfg 033.cfg 216.cfg 614.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 612.out 613.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out 801.out /var/lib/xfstests/tests/xfs 2024-09-21T02:12:26.954 INFO:teuthology.orchestra.run.smithi096.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-09-21T02:12:26.964 INFO:teuthology.orchestra.run.smithi096.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-09-21T02:12:26.974 INFO:teuthology.orchestra.run.smithi096.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-09-21T02:12:26.983 INFO:teuthology.orchestra.run.smithi096.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-09-21T02:12:26.996 INFO:tasks.cephfs.test_acls:client is fuse mounted 2024-09-21T02:12:26.996 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-21T02:12:26.996 DEBUG:teuthology.orchestra.run.smithi096:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.trp2hAHaMUxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-09-21T02:12:27.255 INFO:teuthology.orchestra.run.smithi096.stdout:FSTYP -- ceph-fuse 2024-09-21T02:12:27.260 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM -- Linux/x86_64 smithi096 5.15.0-122-generic #132-Ubuntu SMP Thu Aug 29 13:45:52 UTC 2024 2024-09-21T02:12:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:27.590 INFO:teuthology.orchestra.run.smithi096.stdout:generic/099 0s 2024-09-21T02:12:27.597 INFO:teuthology.orchestra.run.smithi096.stdout:Ran: generic/099 2024-09-21T02:12:27.597 INFO:teuthology.orchestra.run.smithi096.stdout:Passed all 1 tests 2024-09-21T02:12:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:27.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[38609]: fuse finished with error 0 and tester_r 0 2024-09-21T02:12:27.610 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-09-21T02:12:27.610 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph-fuse PLATFORM -- Linux/x86_64 smithi096 5.15.0-122-generic #132-Ubuntu SMP Thu Aug 29 13:45:52 UTC 2024 generic/099 0s Ran: generic/099 Passed all 1 tests 2024-09-21T02:12:27.610 INFO:tasks.cephfs.test_acls:Command stderr - 2024-09-21T02:12:27.610 DEBUG:teuthology.orchestra.run.smithi096:> sudo userdel --force --remove fsgqa 2024-09-21T02:12:27.696 INFO:teuthology.orchestra.run.smithi096.stderr:userdel: fsgqa mail spool (/var/mail/fsgqa) not found 2024-09-21T02:12:28.068 DEBUG:teuthology.orchestra.run.smithi096:> sudo userdel --force --remove 123456-fsgqa 2024-09-21T02:12:28.139 INFO:teuthology.orchestra.run.smithi096.stderr:userdel: 123456-fsgqa mail spool (/var/mail/123456-fsgqa) not found 2024-09-21T02:12:28.139 INFO:teuthology.orchestra.run.smithi096.stderr:userdel: 123456-fsgqa home directory (/home/123456-fsgqa) not found 2024-09-21T02:12:28.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo groupdel fsgqa 2024-09-21T02:12:28.544 INFO:teuthology.orchestra.run.smithi096.stderr:groupdel: group 'fsgqa' does not exist 2024-09-21T02:12:28.545 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T02:12:28.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:28.546 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.trp2hAHaMUxfstests-dev/results) 2024-09-21T02:12:28.590 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-09-21T02:12:28.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:28.591 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.trp2hAHaMUxfstests-dev 2024-09-21T02:12:28.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf /tmp/tmp.trp2hAHaMUxfstests-dev 2024-09-21T02:12:28.719 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:12:28.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:12:28.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:12:28.868 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:12:28.868 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:12:28.868 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:12:28.868 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:28.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20690 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:12:28.875 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:12:28.875 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:12:28.875 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:28.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336312k,nr_inodes=4084078,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16384 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274648k,nr_inodes=818662,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:12:28.881 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:12:28.881 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:12:28.881 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:28.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20690 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:12:28.924 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:12:28.924 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:12:28.924 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:28.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336312k,nr_inodes=4084078,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16384 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274648k,nr_inodes=818662,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:12:28.930 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:12:29.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 -- 172.21.15.96:0/3668328232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face80cec00 msgr2=0x7face80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 --2- 172.21.15.96:0/3668328232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face80cec00 0x7face80cf010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7face4009840 tx=0x7face401c790 comp rx=0 tx=0).stop 2024-09-21T02:12:29.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 -- 172.21.15.96:0/3668328232 shutdown_connections 2024-09-21T02:12:29.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 --2- 172.21.15.96:0/3668328232 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 0x7face80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 --2- 172.21.15.96:0/3668328232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face80cec00 0x7face80cf010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 -- 172.21.15.96:0/3668328232 >> 172.21.15.96:0/3668328232 conn(0x7face806eea0 msgr2=0x7face806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:29.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 -- 172.21.15.96:0/3668328232 shutdown_connections 2024-09-21T02:12:29.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 -- 172.21.15.96:0/3668328232 wait complete. 2024-09-21T02:12:29.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 Processor -- start 2024-09-21T02:12:29.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.127+0000 7facf0653640 1 -- start start 2024-09-21T02:12:29.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 0x7face814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 0x7face8151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7face80cec00 0x7face80b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7face8169c60 con 0x7face806e2c0 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7face8169ae0 con 0x7face80cec00 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7face8169de0 con 0x7face80c4c90 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 0x7face814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 0x7face814f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43508/0 (socket says 172.21.15.96:43508) 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 -- 172.21.15.96:0/169698648 learned_addr learned my addr 172.21.15.96:0/169698648 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facedbee640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 0x7face8151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7faceebf0640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7face80cec00 0x7face80b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 -- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 msgr2=0x7face8151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 0x7face8151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 -- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7face80cec00 msgr2=0x7face80b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7face80cec00 0x7face80b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 -- 172.21.15.96:0/169698648 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80c9f10 con 0x7face806e2c0 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facedbee640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 0x7face8151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7faceebf0640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7face80cec00 0x7face80b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facee3ef640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 0x7face814f610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facd800c410 tx=0x7facd800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8011790 con 0x7face806e2c0 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facd8017070 con 0x7face806e2c0 2024-09-21T02:12:29.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8011ce0 con 0x7face806e2c0 2024-09-21T02:12:29.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face814fb50 con 0x7face806e2c0 2024-09-21T02:12:29.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face80b4b80 con 0x7face806e2c0 2024-09-21T02:12:29.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.131+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face80cf010 con 0x7face806e2c0 2024-09-21T02:12:29.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.135+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7facd8012420 con 0x7face806e2c0 2024-09-21T02:12:29.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.135+0000 7facdeffd640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faccc073230 0x7faccc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.135+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7facd8093be0 con 0x7face806e2c0 2024-09-21T02:12:29.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.135+0000 7facedbee640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faccc073230 0x7faccc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.135+0000 7facedbee640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faccc073230 0x7faccc075740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7face40ae000 tx=0x7face401afd0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.139+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7facd8020020 con 0x7face806e2c0 2024-09-21T02:12:29.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.347+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7face80c6570 con 0x7face806e2c0 2024-09-21T02:12:29.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facdeffd640 1 -- 172.21.15.96:0/169698648 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7facd805ea40 con 0x7face806e2c0 2024-09-21T02:12:29.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:29.355 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"btime":"2024-09-21T02:10:29:323484+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:10:25.710385+0000","modified":"2024-09-21T02:10:28.731053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5037":{"gid":5037,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3851681791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3851681791},{"type":"v1","addr":"172.21.15.96:6835","nonce":3851681791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5037,"qdb_cluster":[5037]},"id":2}]} 2024-09-21T02:12:29.355 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-09-21T02:12:29.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faccc073230 msgr2=0x7faccc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faccc073230 0x7faccc075740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7face40ae000 tx=0x7face401afd0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 msgr2=0x7face814f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 0x7face814f610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facd800c410 tx=0x7facd800c8d0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 shutdown_connections 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7faccc073230 0x7faccc075740 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7face80cec00 0x7face80b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7face80c4c90 0x7face8151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 --2- 172.21.15.96:0/169698648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7face806e2c0 0x7face814f610 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 >> 172.21.15.96:0/169698648 conn(0x7face806eea0 msgr2=0x7face815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 shutdown_connections 2024-09-21T02:12:29.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.351+0000 7facf0653640 1 -- 172.21.15.96:0/169698648 wait complete. 2024-09-21T02:12:29.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:12:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 -- 172.21.15.96:0/545821886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300c4ac0 msgr2=0x7f30300c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 --2- 172.21.15.96:0/545821886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300c4ac0 0x7f30300c4ed0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3018009bc0 tx=0x7f301801cb10 comp rx=0 tx=0).stop 2024-09-21T02:12:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 -- 172.21.15.96:0/545821886 shutdown_connections 2024-09-21T02:12:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 --2- 172.21.15.96:0/545821886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300c4ac0 0x7f30300c4ed0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 --2- 172.21.15.96:0/545821886 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f303006e2c0 0x7f30300c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 -- 172.21.15.96:0/545821886 >> 172.21.15.96:0/545821886 conn(0x7f303006eea0 msgr2=0x7f303006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:29.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 -- 172.21.15.96:0/545821886 shutdown_connections 2024-09-21T02:12:29.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.527+0000 7f30367ca640 1 -- 172.21.15.96:0/545821886 wait complete. 2024-09-21T02:12:29.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 Processor -- start 2024-09-21T02:12:29.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 -- start start 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f303006e2c0 0x7f303014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30300c4ac0 0x7f30300c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 0x7f30300c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30300cf0a0 con 0x7f30300cdbf0 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30300cef20 con 0x7f30300c4ac0 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f30300cf220 con 0x7f303006e2c0 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 0x7f30300c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 0x7f30300c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43516/0 (socket says 172.21.15.96:43516) 2024-09-21T02:12:29.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 -- 172.21.15.96:0/3266150942 learned_addr learned my addr 172.21.15.96:0/3266150942 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f302ffff640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f303006e2c0 0x7f303014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f302f7fe640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30300c4ac0 0x7f30300c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 -- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f303006e2c0 msgr2=0x7f303014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f303006e2c0 0x7f303014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 -- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30300c4ac0 msgr2=0x7f30300c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30300c4ac0 0x7f30300c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 -- 172.21.15.96:0/3266150942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3030150a30 con 0x7f30300cdbf0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f3034d67640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 0x7f30300c7dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f302800c3c0 tx=0x7f302800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028017070 con 0x7f30300cdbf0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303014fb00 con 0x7f30300cdbf0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3028003600 con 0x7f30300cdbf0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028003a50 con 0x7f30300cdbf0 2024-09-21T02:12:29.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.531+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30301500c0 con 0x7f30300cdbf0 2024-09-21T02:12:29.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.535+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30300c4580 con 0x7f30300cdbf0 2024-09-21T02:12:29.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.535+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f302801a070 con 0x7f30300cdbf0 2024-09-21T02:12:29.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.535+0000 7f302cff9640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3010073230 0x7f3010075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.535+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3028093640 con 0x7f30300cdbf0 2024-09-21T02:12:29.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.535+0000 7f302ffff640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3010073230 0x7f3010075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.535+0000 7f302ffff640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3010073230 0x7f3010075740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30200058d0 tx=0x7f3020005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.539+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f302805e550 con 0x7f30300cdbf0 2024-09-21T02:12:29.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.739+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3030073310 con 0x7f30300cdbf0 2024-09-21T02:12:29.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.739+0000 7f302cff9640 1 -- 172.21.15.96:0/3266150942 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f302805de50 con 0x7f30300cdbf0 2024-09-21T02:12:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:29.745 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"btime":"2024-09-21T02:10:29:323484+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:10:25.710385+0000","modified":"2024-09-21T02:10:28.731053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5037":{"gid":5037,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3851681791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3851681791},{"type":"v1","addr":"172.21.15.96:6835","nonce":3851681791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5037,"qdb_cluster":[5037]},"id":2}]} 2024-09-21T02:12:29.745 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3010073230 msgr2=0x7f3010075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3010073230 0x7f3010075740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30200058d0 tx=0x7f3020005860 comp rx=0 tx=0).stop 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 msgr2=0x7f30300c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 0x7f30300c7dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f302800c3c0 tx=0x7f302800c7c0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 shutdown_connections 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f3010073230 0x7f3010075740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30300cdbf0 0x7f30300c7dd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30300c4ac0 0x7f30300c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 --2- 172.21.15.96:0/3266150942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f303006e2c0 0x7f303014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 >> 172.21.15.96:0/3266150942 conn(0x7f303006eea0 msgr2=0x7f30300b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 shutdown_connections 2024-09-21T02:12:29.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.743+0000 7f30367ca640 1 -- 172.21.15.96:0/3266150942 wait complete. 2024-09-21T02:12:29.767 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:12:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 -- 172.21.15.96:0/533415974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415b310 msgr2=0x7ff2540cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 --2- 172.21.15.96:0/533415974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415b310 0x7ff2540cebe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff254073ce0 tx=0x7ff25001c890 comp rx=0 tx=0).stop 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 -- 172.21.15.96:0/533415974 shutdown_connections 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 --2- 172.21.15.96:0/533415974 >> v2:172.21.15.122:3300/0 conn(0x7ff25415f750 0x7ff2540cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 --2- 172.21.15.96:0/533415974 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415fb60 0x7ff2540d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 --2- 172.21.15.96:0/533415974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415b310 0x7ff2540cebe0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 -- 172.21.15.96:0/533415974 >> 172.21.15.96:0/533415974 conn(0x7ff25406fbe0 msgr2=0x7ff25406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 -- 172.21.15.96:0/533415974 shutdown_connections 2024-09-21T02:12:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 -- 172.21.15.96:0/533415974 wait complete. 2024-09-21T02:12:29.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 Processor -- start 2024-09-21T02:12:29.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.903+0000 7ff25bf57640 1 -- start start 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415b310 0x7ff254147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 0x7ff2540b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff25415fb60 0x7ff2540b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff25406e330 con 0x7ff25415f750 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff25406e1b0 con 0x7ff25415fb60 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff25406e4b0 con 0x7ff25415b310 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 0x7ff2540b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 0x7ff2540b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43538/0 (socket says 172.21.15.96:43538) 2024-09-21T02:12:29.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 -- 172.21.15.96:0/3455323133 learned_addr learned my addr 172.21.15.96:0/3455323133 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25a4f4640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff25415fb60 0x7ff2540b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff259cf3640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415b310 0x7ff254147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 -- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415b310 msgr2=0x7ff254147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415b310 0x7ff254147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 -- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff25415fb60 msgr2=0x7ff2540b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff25415fb60 0x7ff2540b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 -- 172.21.15.96:0/3455323133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff254147940 con 0x7ff25415f750 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff259cf3640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415b310 0x7ff254147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25a4f4640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff25415fb60 0x7ff2540b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff2594f2640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 0x7ff2540b6520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff244005950 tx=0x7ff24400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244007bb0 con 0x7ff25415f750 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff244007e40 con 0x7ff25415f750 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2541f6720 con 0x7ff25415f750 2024-09-21T02:12:29.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400f690 con 0x7ff25415f750 2024-09-21T02:12:29.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2541f6ce0 con 0x7ff25415f750 2024-09-21T02:12:29.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2540cebc0 con 0x7ff25415f750 2024-09-21T02:12:29.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff24400f830 con 0x7ff25415f750 2024-09-21T02:12:29.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff24a7fc640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff238073200 0x7ff238075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:29.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff244094b50 con 0x7ff25415f750 2024-09-21T02:12:29.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.907+0000 7ff259cf3640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff238073200 0x7ff238075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:29.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.911+0000 7ff259cf3640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff238073200 0x7ff238075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2500af000 tx=0x7ff25001b0d0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:29.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:29.911+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff244014030 con 0x7ff25415f750 2024-09-21T02:12:30.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.075+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff254069130 con 0x7ff25415f750 2024-09-21T02:12:30.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.075+0000 7ff24a7fc640 1 -- 172.21.15.96:0/3455323133 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16995 (secure 0 0 0) 0x7ff24405f9c0 con 0x7ff25415f750 2024-09-21T02:12:30.080 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:30.080 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:10:52.635532+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:10:23.405387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T02:10:24.661586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.36","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.075+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff238073200 msgr2=0x7ff238075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.075+0000 7ff25bf57640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff238073200 0x7ff238075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2500af000 tx=0x7ff25001b0d0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.075+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 msgr2=0x7ff2540b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.075+0000 7ff25bf57640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 0x7ff2540b6520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff244005950 tx=0x7ff24400cdd0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 shutdown_connections 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7ff238073200 0x7ff238075710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff25415fb60 0x7ff2540b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff25415f750 0x7ff2540b6520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 --2- 172.21.15.96:0/3455323133 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff25415b310 0x7ff254147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 >> 172.21.15.96:0/3455323133 conn(0x7ff25406fbe0 msgr2=0x7ff25406e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:30.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 shutdown_connections 2024-09-21T02:12:30.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.079+0000 7ff25bf57640 1 -- 172.21.15.96:0/3455323133 wait complete. 2024-09-21T02:12:30.102 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:12:30.102 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:12:30.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 -- 172.21.15.96:0/1927259486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec48004010 msgr2=0x7fec48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 --2- 172.21.15.96:0/1927259486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec48004010 0x7fec48004810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c290 tx=0x7fec4c01f6c0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec61823640 1 -- 172.21.15.96:0/1927259486 reap_dead start 2024-09-21T02:12:30.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 -- 172.21.15.96:0/1927259486 shutdown_connections 2024-09-21T02:12:30.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 -- 172.21.15.96:0/1927259486 >> 172.21.15.96:0/1927259486 conn(0x7fec5c06eea0 msgr2=0x7fec5c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:30.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 -- 172.21.15.96:0/1927259486 shutdown_connections 2024-09-21T02:12:30.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 -- 172.21.15.96:0/1927259486 wait complete. 2024-09-21T02:12:30.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 Processor -- start 2024-09-21T02:12:30.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.247+0000 7fec63286640 1 -- start start 2024-09-21T02:12:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec5c15ec90 0x7fec5c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec5c0cec60 0x7fec5c140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 0x7fec5c146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c06fd50 con 0x7fec5c141020 2024-09-21T02:12:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c06fbd0 con 0x7fec5c0cec60 2024-09-21T02:12:30.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c06fed0 con 0x7fec5c15ec90 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 0x7fec5c146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 0x7fec5c146c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43560/0 (socket says 172.21.15.96:43560) 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 -- 172.21.15.96:0/846434205 learned_addr learned my addr 172.21.15.96:0/846434205 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61022640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec5c15ec90 0x7fec5c0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec60821640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec5c0cec60 0x7fec5c140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 -- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec5c15ec90 msgr2=0x7fec5c0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec5c15ec90 0x7fec5c0ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 -- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec5c0cec60 msgr2=0x7fec5c140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec5c0cec60 0x7fec5c140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 -- 172.21.15.96:0/846434205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c0b0070 con 0x7fec5c141020 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61022640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec5c15ec90 0x7fec5c0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec61823640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 0x7fec5c146c40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0cdba0 tx=0x7fec5400eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5400f040 con 0x7fec5c141020 2024-09-21T02:12:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec54018070 con 0x7fec5c141020 2024-09-21T02:12:30.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c0cf070 con 0x7fec5c141020 2024-09-21T02:12:30.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5400c900 con 0x7fec5c141020 2024-09-21T02:12:30.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c15e1d0 con 0x7fec5c141020 2024-09-21T02:12:30.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5c0cab60 con 0x7fec5c141020 2024-09-21T02:12:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fec5400caa0 con 0x7fec5c141020 2024-09-21T02:12:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec51ffb640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fec40073200 0x7fec40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.251+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fec54093e80 con 0x7fec5c141020 2024-09-21T02:12:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.255+0000 7fec61022640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fec40073200 0x7fec40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.255+0000 7fec61022640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fec40073200 0x7fec40075710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec5c1eb830 tx=0x7fec58005750 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.259+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec5c0cf070 con 0x7fec5c141020 2024-09-21T02:12:30.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.451+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec5c06e9d0 con 0x7fec5c141020 2024-09-21T02:12:30.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.451+0000 7fec51ffb640 1 -- 172.21.15.96:0/846434205 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7fec5405ed80 con 0x7fec5c141020 2024-09-21T02:12:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:30.457 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"btime":"2024-09-21T02:10:29:323484+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:10:25.710385+0000","modified":"2024-09-21T02:10:28.731053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5037":{"gid":5037,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3851681791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3851681791},{"type":"v1","addr":"172.21.15.96:6835","nonce":3851681791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5037,"qdb_cluster":[5037]},"id":2}]} 2024-09-21T02:12:30.457 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-09-21T02:12:30.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fec40073200 msgr2=0x7fec40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fec40073200 0x7fec40075710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec5c1eb830 tx=0x7fec58005750 comp rx=0 tx=0).stop 2024-09-21T02:12:30.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 msgr2=0x7fec5c146c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 0x7fec5c146c40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0cdba0 tx=0x7fec5400eb60 comp rx=0 tx=0).stop 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 shutdown_connections 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fec40073200 0x7fec40075710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5c141020 0x7fec5c146c40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec5c0cec60 0x7fec5c140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 --2- 172.21.15.96:0/846434205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec5c15ec90 0x7fec5c0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 >> 172.21.15.96:0/846434205 conn(0x7fec5c06eea0 msgr2=0x7fec5c15a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 shutdown_connections 2024-09-21T02:12:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.455+0000 7fec63286640 1 -- 172.21.15.96:0/846434205 wait complete. 2024-09-21T02:12:30.479 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:12:30.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 -- 172.21.15.96:0/3083637644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806dea0 msgr2=0x7f477806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 --2- 172.21.15.96:0/3083637644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806dea0 0x7f477806e2b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f47780cdb80 tx=0x7f476801c7d0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 -- 172.21.15.96:0/3083637644 shutdown_connections 2024-09-21T02:12:30.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 --2- 172.21.15.96:0/3083637644 >> v2:172.21.15.122:3300/0 conn(0x7f47780cab60 0x7f47780caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 --2- 172.21.15.96:0/3083637644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806dea0 0x7f477806e2b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 -- 172.21.15.96:0/3083637644 >> 172.21.15.96:0/3083637644 conn(0x7f477806eea0 msgr2=0x7f477806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:30.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 -- 172.21.15.96:0/3083637644 shutdown_connections 2024-09-21T02:12:30.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.623+0000 7f477dad5640 1 -- 172.21.15.96:0/3083637644 wait complete. 2024-09-21T02:12:30.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 Processor -- start 2024-09-21T02:12:30.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 -- start start 2024-09-21T02:12:30.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f477806dea0 0x7f477814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47780cab60 0x7f477815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 0x7f47780b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47780cbda0 con 0x7f4778155240 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47780cbc20 con 0x7f477806dea0 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f47780cbf20 con 0x7f47780cab60 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 0x7f47780b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 0x7f47780b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43588/0 (socket says 172.21.15.96:43588) 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 -- 172.21.15.96:0/2658833688 learned_addr learned my addr 172.21.15.96:0/2658833688 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f476ffff640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47780cab60 0x7f477815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f47777fe640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f477806dea0 0x7f477814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 -- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47780cab60 msgr2=0x7f477815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47780cab60 0x7f477815a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 -- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f477806dea0 msgr2=0x7f477814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f477806dea0 0x7f477814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 -- 172.21.15.96:0/2658833688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477814f990 con 0x7f4778155240 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f47777fe640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f477806dea0 0x7f477814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4777fff640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 0x7f47780b4590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f477000ddf0 tx=0x7f477000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770004d30 con 0x7f4778155240 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f477000be40 con 0x7f4778155240 2024-09-21T02:12:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781f9c90 con 0x7f4778155240 2024-09-21T02:12:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477000e8c0 con 0x7f4778155240 2024-09-21T02:12:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.627+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781fa250 con 0x7f4778155240 2024-09-21T02:12:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.631+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477806e2b0 con 0x7f4778155240 2024-09-21T02:12:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.631+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f477000ea60 con 0x7f4778155240 2024-09-21T02:12:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.631+0000 7f4774ff9640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4758073200 0x7f4758075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.631+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f4770094e30 con 0x7f4778155240 2024-09-21T02:12:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.631+0000 7f47777fe640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4758073200 0x7f4758075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.631+0000 7f47777fe640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4758073200 0x7f4758075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4760001630 tx=0x7f4760009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:30.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.639+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f477814f990 con 0x7f4778155240 2024-09-21T02:12:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.839+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477815aef0 con 0x7f4778155240 2024-09-21T02:12:30.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f4774ff9640 1 -- 172.21.15.96:0/2658833688 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f477005fd30 con 0x7f4778155240 2024-09-21T02:12:30.847 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:30.848 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"btime":"2024-09-21T02:10:29:323484+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:10:25.710385+0000","modified":"2024-09-21T02:10:28.731053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5037":{"gid":5037,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3851681791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3851681791},{"type":"v1","addr":"172.21.15.96:6835","nonce":3851681791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5037,"qdb_cluster":[5037]},"id":2}]} 2024-09-21T02:12:30.848 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-09-21T02:12:30.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4758073200 msgr2=0x7f4758075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4758073200 0x7f4758075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4760001630 tx=0x7f4760009040 comp rx=0 tx=0).stop 2024-09-21T02:12:30.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 msgr2=0x7f47780b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:30.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 0x7f47780b4590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f477000ddf0 tx=0x7f477000b730 comp rx=0 tx=0).stop 2024-09-21T02:12:30.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 shutdown_connections 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f4758073200 0x7f4758075710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778155240 0x7f47780b4590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f47780cab60 0x7f477815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 --2- 172.21.15.96:0/2658833688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f477806dea0 0x7f477814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 >> 172.21.15.96:0/2658833688 conn(0x7f477806eea0 msgr2=0x7f477806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 shutdown_connections 2024-09-21T02:12:30.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:30.843+0000 7f477dad5640 1 -- 172.21.15.96:0/2658833688 wait complete. 2024-09-21T02:12:30.869 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:12:31.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9344736640 1 -- 172.21.15.96:0/3837012695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 msgr2=0x7f9328004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9344736640 1 --2- 172.21.15.96:0/3837012695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f9328004810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f932c00bb10 tx=0x7f932c01cdc0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9342cd3640 1 -- 172.21.15.96:0/3837012695 reap_dead start 2024-09-21T02:12:31.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9344736640 1 -- 172.21.15.96:0/3837012695 shutdown_connections 2024-09-21T02:12:31.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9344736640 1 -- 172.21.15.96:0/3837012695 >> 172.21.15.96:0/3837012695 conn(0x7f933c06fbe0 msgr2=0x7f933c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:31.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9344736640 1 -- 172.21.15.96:0/3837012695 shutdown_connections 2024-09-21T02:12:31.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.015+0000 7f9344736640 1 -- 172.21.15.96:0/3837012695 wait complete. 2024-09-21T02:12:31.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 Processor -- start 2024-09-21T02:12:31.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 -- start start 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f933c14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f933c155ea0 0x7f933c1f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f933c1f8ee0 0x7f933c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f933c0d00a0 con 0x7f93280040e0 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f933c0cff20 con 0x7f933c155ea0 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f933c0d0220 con 0x7f933c1f8ee0 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f933c14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f933c14f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43602/0 (socket says 172.21.15.96:43602) 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 -- 172.21.15.96:0/2790662091 learned_addr learned my addr 172.21.15.96:0/2790662091 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:31.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9341cd1640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f933c155ea0 0x7f933c1f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9342cd3640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f933c1f8ee0 0x7f933c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 -- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f933c1f8ee0 msgr2=0x7f933c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f933c1f8ee0 0x7f933c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 -- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f933c155ea0 msgr2=0x7f933c1f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f933c155ea0 0x7f933c1f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 -- 172.21.15.96:0/2790662091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c0b0890 con 0x7f93280040e0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9342cd3640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f933c1f8ee0 0x7f933c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f93424d2640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f933c14f580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9338003f20 tx=0x7f9338001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338011640 con 0x7f93280040e0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9338016020 con 0x7f93280040e0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c1f92f0 con 0x7f93280040e0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338011a80 con 0x7f93280040e0 2024-09-21T02:12:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.019+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1590d0 con 0x7f93280040e0 2024-09-21T02:12:31.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.023+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f933c0b0a60 con 0x7f93280040e0 2024-09-21T02:12:31.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.023+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9338011c20 con 0x7f93280040e0 2024-09-21T02:12:31.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.023+0000 7f9332ffd640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9320073200 0x7f9320075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.023+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9338095090 con 0x7f93280040e0 2024-09-21T02:12:31.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.023+0000 7f9341cd1640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9320073200 0x7f9320075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.023+0000 7f9341cd1640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9320073200 0x7f9320075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f932c0b0000 tx=0x7f9328005840 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:31.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.027+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9338020020 con 0x7f93280040e0 2024-09-21T02:12:31.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.203+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f933c06db60 con 0x7f93280040e0 2024-09-21T02:12:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.203+0000 7f9332ffd640 1 -- 172.21.15.96:0/2790662091 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16995 (secure 0 0 0) 0x7f933805ff90 con 0x7f93280040e0 2024-09-21T02:12:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:31.210 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:10:52.635532+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:10:23.405387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T02:10:24.661586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.36","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:12:31.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9320073200 msgr2=0x7f9320075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9320073200 0x7f9320075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f932c0b0000 tx=0x7f9328005840 comp rx=0 tx=0).stop 2024-09-21T02:12:31.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 msgr2=0x7f933c14f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f933c14f580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9338003f20 tx=0x7f9338001700 comp rx=0 tx=0).stop 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 shutdown_connections 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f9320073200 0x7f9320075710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f933c1f8ee0 0x7f933c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f933c155ea0 0x7f933c1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 --2- 172.21.15.96:0/2790662091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93280040e0 0x7f933c14f580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 >> 172.21.15.96:0/2790662091 conn(0x7f933c06fbe0 msgr2=0x7f933c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 shutdown_connections 2024-09-21T02:12:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.207+0000 7f9344736640 1 -- 172.21.15.96:0/2790662091 wait complete. 2024-09-21T02:12:31.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- 172.21.15.96:0/571443666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d006e5b0 msgr2=0x7fd3d00cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 --2- 172.21.15.96:0/571443666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d006e5b0 0x7fd3d00cab60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00cdb80 tx=0x7fd3c801cd00 comp rx=0 tx=0).stop 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- 172.21.15.96:0/571443666 shutdown_connections 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 --2- 172.21.15.96:0/571443666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d006e5b0 0x7fd3d00cab60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 --2- 172.21.15.96:0/571443666 >> v2:172.21.15.122:3300/0 conn(0x7fd3d0155240 0x7fd3d0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- 172.21.15.96:0/571443666 >> 172.21.15.96:0/571443666 conn(0x7fd3d006eea0 msgr2=0x7fd3d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- 172.21.15.96:0/571443666 shutdown_connections 2024-09-21T02:12:31.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- 172.21.15.96:0/571443666 wait complete. 2024-09-21T02:12:31.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 Processor -- start 2024-09-21T02:12:31.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- start start 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3d0073970 0x7fd3d00c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 0x7fd3d00b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d014f880 0x7fd3d00b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d00cbe20 con 0x7fd3d0155240 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d00cbca0 con 0x7fd3d014f880 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3d5578640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d0168c80 con 0x7fd3d0073970 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 0x7fd3d00b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 0x7fd3d00b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43632/0 (socket says 172.21.15.96:43632) 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 -- 172.21.15.96:0/1783719297 learned_addr learned my addr 172.21.15.96:0/1783719297 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 -- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3d0073970 msgr2=0x7fd3d00c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3d0073970 0x7fd3d00c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 -- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d014f880 msgr2=0x7fd3d00b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d014f880 0x7fd3d00b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.375+0000 7fd3c67fc640 1 -- 172.21.15.96:0/1783719297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d00c3b50 con 0x7fd3d0155240 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c67fc640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 0x7fd3d00b3be0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b410 tx=0x7fd3b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8015030 con 0x7fd3d0155240 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b8004020 con 0x7fd3d0155240 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d00c6950 con 0x7fd3d0155240 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8004470 con 0x7fd3d0155240 2024-09-21T02:12:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d00c9a20 con 0x7fd3d0155240 2024-09-21T02:12:31.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d00cab60 con 0x7fd3d0155240 2024-09-21T02:12:31.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd3b8018070 con 0x7fd3d0155240 2024-09-21T02:12:31.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c7fff640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd3b0073200 0x7fd3b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd3b8094310 con 0x7fd3d0155240 2024-09-21T02:12:31.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3ceffd640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd3b0073200 0x7fd3b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.379+0000 7fd3ceffd640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd3b0073200 0x7fd3b0075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000a910 tx=0x7fd3c0005be0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:31.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.383+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd3b805f2f0 con 0x7fd3d0155240 2024-09-21T02:12:31.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.587+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd3d006e9d0 con 0x7fd3d0155240 2024-09-21T02:12:31.595 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-21T02:12:31.591+0000 7f19643c0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:12:31.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.591+0000 7fd3c7fff640 1 -- 172.21.15.96:0/1783719297 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fd3b805ebf0 con 0x7fd3d0155240 2024-09-21T02:12:31.598 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd3b0073200 msgr2=0x7fd3b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd3b0073200 0x7fd3b0075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000a910 tx=0x7fd3c0005be0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 msgr2=0x7fd3d00b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 0x7fd3d00b3be0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b410 tx=0x7fd3b800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 shutdown_connections 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fd3b0073200 0x7fd3b0075710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d014f880 0x7fd3d00b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d0155240 0x7fd3d00b3be0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 --2- 172.21.15.96:0/1783719297 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3d0073970 0x7fd3d00c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 >> 172.21.15.96:0/1783719297 conn(0x7fd3d006eea0 msgr2=0x7fd3d00cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:31.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 shutdown_connections 2024-09-21T02:12:31.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.595+0000 7fd3d5578640 1 -- 172.21.15.96:0/1783719297 wait complete. 2024-09-21T02:12:31.606 INFO:tasks.ceph.mds.a.smithi096.stderr: -7549> 2024-09-21T02:10:29.751+0000 7f821d9f8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:12:31.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:12:31.666 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-21T02:12:31.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 -- 172.21.15.96:0/3074080867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d8073380 msgr2=0x7f96d8073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 --2- 172.21.15.96:0/3074080867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d8073380 0x7f96d8073790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009900 tx=0x7f96cc01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 -- 172.21.15.96:0/3074080867 shutdown_connections 2024-09-21T02:12:31.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 --2- 172.21.15.96:0/3074080867 >> v2:172.21.15.122:3300/0 conn(0x7f96d80ce000 0x7f96d80ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 --2- 172.21.15.96:0/3074080867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d8073380 0x7f96d8073790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 -- 172.21.15.96:0/3074080867 >> 172.21.15.96:0/3074080867 conn(0x7f96d80b1a30 msgr2=0x7f96d80b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 -- 172.21.15.96:0/3074080867 shutdown_connections 2024-09-21T02:12:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 -- 172.21.15.96:0/3074080867 wait complete. 2024-09-21T02:12:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 Processor -- start 2024-09-21T02:12:31.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.763+0000 7f96dd014640 1 -- start start 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d8073380 0x7f96d81f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d80ce000 0x7f96d814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 0x7f96d814abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81659b0 con 0x7f96d80cf5d0 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8165830 con 0x7f96d8073380 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f96d8165b30 con 0x7f96d80ce000 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 0x7f96d814abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 0x7f96d814abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43658/0 (socket says 172.21.15.96:43658) 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 -- 172.21.15.96:0/874173739 learned_addr learned my addr 172.21.15.96:0/874173739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d7fff640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d8073380 0x7f96d81f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d77fe640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d80ce000 0x7f96d814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 -- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d80ce000 msgr2=0x7f96d814d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d80ce000 0x7f96d814d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 -- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d8073380 msgr2=0x7f96d81f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d8073380 0x7f96d81f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 -- 172.21.15.96:0/874173739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d81f80e0 con 0x7f96d80cf5d0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d77fe640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d80ce000 0x7f96d814d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dc813640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 0x7f96d814abe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ec00 tx=0x7f96d000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0003b80 con 0x7f96d80cf5d0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96d0003e10 con 0x7f96d80cf5d0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d81f85f0 con 0x7f96d80cf5d0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0004280 con 0x7f96d80cf5d0 2024-09-21T02:12:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d814a3e0 con 0x7f96d80cf5d0 2024-09-21T02:12:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d81f8360 con 0x7f96d80cf5d0 2024-09-21T02:12:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96d0004420 con 0x7f96d80cf5d0 2024-09-21T02:12:31.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d4ff9640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f96c4073200 0x7f96c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.767+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7f96d0095380 con 0x7f96d80cf5d0 2024-09-21T02:12:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.771+0000 7f96d7fff640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f96c4073200 0x7f96c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:31.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.771+0000 7f96d7fff640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f96c4073200 0x7f96c4075710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f96c8001650 tx=0x7f96c8009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:31.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.775+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f96d0060120 con 0x7f96d80cf5d0 2024-09-21T02:12:31.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:31.979+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f96d8073790 con 0x7f96d80cf5d0 2024-09-21T02:12:32.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.643+0000 7f96d4ff9640 1 -- 172.21.15.96:0/874173739 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f96d005fa20 con 0x7f96d80cf5d0 2024-09-21T02:12:32.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f96c4073200 msgr2=0x7f96c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:32.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f96c4073200 0x7f96c4075710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f96c8001650 tx=0x7f96c8009040 comp rx=0 tx=0).stop 2024-09-21T02:12:32.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 msgr2=0x7f96d814abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:32.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 0x7f96d814abe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ec00 tx=0x7f96d000c6a0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 shutdown_connections 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f96c4073200 0x7f96c4075710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96d80cf5d0 0x7f96d814abe0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96d80ce000 0x7f96d814d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 --2- 172.21.15.96:0/874173739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d8073380 0x7f96d81f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 >> 172.21.15.96:0/874173739 conn(0x7f96d80b1a30 msgr2=0x7f96d815b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 shutdown_connections 2024-09-21T02:12:32.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.647+0000 7f96dd014640 1 -- 172.21.15.96:0/874173739 wait complete. 2024-09-21T02:12:32.681 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:12:32.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 -- 172.21.15.96:0/4040438107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d0006f700 msgr2=0x7f8d0006fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:32.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 --2- 172.21.15.96:0/4040438107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d0006f700 0x7f8d0006fb10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009900 tx=0x7f8cf001c780 comp rx=0 tx=0).stop 2024-09-21T02:12:32.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 -- 172.21.15.96:0/4040438107 shutdown_connections 2024-09-21T02:12:32.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 --2- 172.21.15.96:0/4040438107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d0006f700 0x7f8d0006fb10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 --2- 172.21.15.96:0/4040438107 >> v2:172.21.15.122:3300/0 conn(0x7f8d000b2ef0 0x7f8d000cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 -- 172.21.15.96:0/4040438107 >> 172.21.15.96:0/4040438107 conn(0x7f8d000d05a0 msgr2=0x7f8d000d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:32.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 -- 172.21.15.96:0/4040438107 shutdown_connections 2024-09-21T02:12:32.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 -- 172.21.15.96:0/4040438107 wait complete. 2024-09-21T02:12:32.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 Processor -- start 2024-09-21T02:12:32.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.835+0000 7f8d0848b640 1 -- start start 2024-09-21T02:12:32.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d0006f700 0x7f8d001430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 0x7f8d001fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d000cbed0 0x7f8d0013dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00073740 con 0x7f8d000b2ef0 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000735c0 con 0x7f8d000cbed0 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8d000738c0 con 0x7f8d0006f700 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 0x7f8d001fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 0x7f8d001fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43680/0 (socket says 172.21.15.96:43680) 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 -- 172.21.15.96:0/1248893329 learned_addr learned my addr 172.21.15.96:0/1248893329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d06227640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d0006f700 0x7f8d001430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d06a28640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d000cbed0 0x7f8d0013dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 -- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d0006f700 msgr2=0x7f8d001430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:32.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d0006f700 0x7f8d001430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 -- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d000cbed0 msgr2=0x7f8d0013dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d000cbed0 0x7f8d0013dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 -- 172.21.15.96:0/1248893329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d00143730 con 0x7f8d000b2ef0 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d05a26640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 0x7f8d001fc660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0019390 tx=0x7f8cf00a8560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0004300 con 0x7f8d000b2ef0 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cf0005070 con 0x7f8d000b2ef0 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d00153410 con 0x7f8d000b2ef0 2024-09-21T02:12:32.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00054c0 con 0x7f8d000b2ef0 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d001539d0 con 0x7f8d000b2ef0 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0006fb10 con 0x7f8d000b2ef0 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8cf00046d0 con 0x7f8d000b2ef0 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8cf6ffd640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ce4073230 0x7f8ce4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7f8cf012e4e0 con 0x7f8d000b2ef0 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.839+0000 7f8d06227640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ce4073230 0x7f8ce4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:32.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.843+0000 7f8d06227640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ce4073230 0x7f8ce4075740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0058d0 tx=0x7f8cfc005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:32.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:32.847+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8cf00ac030 con 0x7f8d000b2ef0 2024-09-21T02:12:33.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.003+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d0013e140 con 0x7f8d000b2ef0 2024-09-21T02:12:33.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.655+0000 7f8cf6ffd640 1 -- 172.21.15.96:0/1248893329 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) ==== 181+0+0 (secure 0 0 0) 0x7f8cf0016490 con 0x7f8d000b2ef0 2024-09-21T02:12:33.662 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:12:33.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ce4073230 msgr2=0x7f8ce4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ce4073230 0x7f8ce4075740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0058d0 tx=0x7f8cfc005860 comp rx=0 tx=0).stop 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 msgr2=0x7f8d001fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 0x7f8d001fc660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0019390 tx=0x7f8cf00a8560 comp rx=0 tx=0).stop 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 shutdown_connections 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8ce4073230 0x7f8ce4075740 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d000cbed0 0x7f8d0013dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d000b2ef0 0x7f8d001fc660 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 --2- 172.21.15.96:0/1248893329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d0006f700 0x7f8d001430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 >> 172.21.15.96:0/1248893329 conn(0x7f8d000d05a0 msgr2=0x7f8d000744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 shutdown_connections 2024-09-21T02:12:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.659+0000 7f8d0848b640 1 -- 172.21.15.96:0/1248893329 wait complete. 2024-09-21T02:12:33.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- 172.21.15.96:0/3858295288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 msgr2=0x7f256c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 --2- 172.21.15.96:0/3858295288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f256c00ba60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2580075030 tx=0x7f257801d150 comp rx=0 tx=0).stop 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- 172.21.15.96:0/3858295288 shutdown_connections 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 --2- 172.21.15.96:0/3858295288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f256c00ba60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- 172.21.15.96:0/3858295288 >> 172.21.15.96:0/3858295288 conn(0x7f25800d05a0 msgr2=0x7f25800d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- 172.21.15.96:0/3858295288 shutdown_connections 2024-09-21T02:12:33.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- 172.21.15.96:0/3858295288 wait complete. 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 Processor -- start 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- start start 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f25800ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25801f5d70 0x7f25801f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f258015db40 0x7f258015df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f258015f960 con 0x7f256c0053b0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f258015f7e0 con 0x7f258015db40 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2585445640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f25800b2ef0 con 0x7f25801f5d70 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f257effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f25800ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f257effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f25800ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43696/0 (socket says 172.21.15.96:43696) 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f257effd640 1 -- 172.21.15.96:0/1983375672 learned_addr learned my addr 172.21.15.96:0/1983375672 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f2577fff640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25801f5d70 0x7f25801f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.855+0000 7f257f7fe640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f258015db40 0x7f258015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f257effd640 1 -- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25801f5d70 msgr2=0x7f25801f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f257effd640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25801f5d70 0x7f25801f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f257effd640 1 -- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f258015db40 msgr2=0x7f258015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f257effd640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f258015db40 0x7f258015df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f257effd640 1 -- 172.21.15.96:0/1983375672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258015e490 con 0x7f256c0053b0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f257effd640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f25800ca9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f257000d6c0 tx=0x7f257000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25700060d0 con 0x7f256c0053b0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2570015020 con 0x7f256c0053b0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257000b7d0 con 0x7f256c0053b0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25800c93f0 con 0x7f256c0053b0 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25800c99b0 con 0x7f256c0053b0 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25800b3450 con 0x7f256c0053b0 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f257000b970 con 0x7f256c0053b0 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f25777fe640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2560073200 0x7f2560075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5778+0+0 (secure 0 0 0) 0x7f2570094880 con 0x7f256c0053b0 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.859+0000 7f2577fff640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2560073200 0x7f2560075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.863+0000 7f2577fff640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2560073200 0x7f2560075710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2568000fe0 tx=0x7f2568001cc0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:33.863+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f257001f020 con 0x7f256c0053b0 2024-09-21T02:12:34.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.035+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25800d0b50 con 0x7f256c0053b0 2024-09-21T02:12:34.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.683+0000 7f25777fe640 1 -- 172.21.15.96:0/1983375672 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7f258015e490 con 0x7f256c0053b0 2024-09-21T02:12:34.690 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2560073200 msgr2=0x7f2560075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2560073200 0x7f2560075710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2568000fe0 tx=0x7f2568001cc0 comp rx=0 tx=0).stop 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 msgr2=0x7f25800ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f25800ca9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f257000d6c0 tx=0x7f257000db80 comp rx=0 tx=0).stop 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 shutdown_connections 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f2560073200 0x7f2560075710 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2568000fe0 tx=0x7f2568001cc0 comp rx=0 tx=0).stop 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f258015db40 0x7f258015df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25801f5d70 0x7f25801f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 --2- 172.21.15.96:0/1983375672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f256c0053b0 0x7f25800ca9c0 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f257000d6c0 tx=0x7f257000db80 comp rx=0 tx=0).stop 2024-09-21T02:12:34.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 >> 172.21.15.96:0/1983375672 conn(0x7f25800d05a0 msgr2=0x7f25801582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:34.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 shutdown_connections 2024-09-21T02:12:34.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.691+0000 7f2585445640 1 -- 172.21.15.96:0/1983375672 wait complete. 2024-09-21T02:12:34.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:12:34.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- 172.21.15.96:0/2972337297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b8004d50 msgr2=0x7f35b8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 --2- 172.21.15.96:0/2972337297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b8004d50 0x7f35b8005200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f35c4001d60 tx=0x7f35c401c940 comp rx=0 tx=0).stop 2024-09-21T02:12:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 -- 172.21.15.96:0/2972337297 reap_dead start 2024-09-21T02:12:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- 172.21.15.96:0/2972337297 shutdown_connections 2024-09-21T02:12:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- 172.21.15.96:0/2972337297 >> 172.21.15.96:0/2972337297 conn(0x7f35cc06eea0 msgr2=0x7f35cc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- 172.21.15.96:0/2972337297 shutdown_connections 2024-09-21T02:12:34.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- 172.21.15.96:0/2972337297 wait complete. 2024-09-21T02:12:34.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 Processor -- start 2024-09-21T02:12:34.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- start start 2024-09-21T02:12:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35cc145010 0x7f35cc0c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35cc156600 0x7f35cc144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 0x7f35cc144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc06e020 con 0x7f35cc156a10 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc06dea0 con 0x7f35cc156600 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d3bee640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc06e1a0 con 0x7f35cc145010 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 0x7f35cc144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 0x7f35cc144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43710/0 (socket says 172.21.15.96:43710) 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 -- 172.21.15.96:0/1894754922 learned_addr learned my addr 172.21.15.96:0/1894754922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d198a640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35cc145010 0x7f35cc0c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d1189640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35cc156600 0x7f35cc144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 -- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35cc145010 msgr2=0x7f35cc0c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35cc145010 0x7f35cc0c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 -- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35cc156600 msgr2=0x7f35cc144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35cc156600 0x7f35cc144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.887+0000 7f35d218b640 1 -- 172.21.15.96:0/1894754922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc0c3660 con 0x7f35cc156a10 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35d218b640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 0x7f35cc144860 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f35c40ab040 tx=0x7f35c4003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4006cb0 con 0x7f35cc156a10 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35c40a5c40 con 0x7f35cc156a10 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc15b730 con 0x7f35cc156a10 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4001990 con 0x7f35cc156a10 2024-09-21T02:12:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc15b450 con 0x7f35cc156a10 2024-09-21T02:12:34.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc075b80 con 0x7f35cc156a10 2024-09-21T02:12:34.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f35c4006070 con 0x7f35cc156a10 2024-09-21T02:12:34.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35c27fc640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f35b0073200 0x7f35b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:34.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f35c412df60 con 0x7f35cc156a10 2024-09-21T02:12:34.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35d198a640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f35b0073200 0x7f35b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.891+0000 7f35d198a640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f35b0073200 0x7f35b0075710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35c80057d0 tx=0x7f35c8009660 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:34.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:34.895+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f35cc15c9c0 con 0x7f35cc156a10 2024-09-21T02:12:35.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.071+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f35cc0b0e10 con 0x7f35cc156a10 2024-09-21T02:12:35.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35c27fc640 1 -- 172.21.15.96:0/1894754922 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f35c40f92f0 con 0x7f35cc156a10 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f35b0073200 msgr2=0x7f35b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f35b0073200 0x7f35b0075710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35c80057d0 tx=0x7f35c8009660 comp rx=0 tx=0).stop 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 msgr2=0x7f35cc144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 0x7f35cc144860 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f35c40ab040 tx=0x7f35c4003ef0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 shutdown_connections 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f35b0073200 0x7f35b0075710 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35cc156a10 0x7f35cc144860 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35cc156600 0x7f35cc144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 --2- 172.21.15.96:0/1894754922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35cc145010 0x7f35cc0c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 >> 172.21.15.96:0/1894754922 conn(0x7f35cc06eea0 msgr2=0x7f35cc073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 shutdown_connections 2024-09-21T02:12:35.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.079+0000 7f35d3bee640 1 -- 172.21.15.96:0/1894754922 wait complete. 2024-09-21T02:12:35.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-09-21T02:12:35.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 -- 172.21.15.96:0/2831500184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec06e2c0 msgr2=0x7effec0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 --2- 172.21.15.96:0/2831500184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec06e2c0 0x7effec0c4580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7effe4009bc0 tx=0x7effe401c9a0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 -- 172.21.15.96:0/2831500184 shutdown_connections 2024-09-21T02:12:35.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 --2- 172.21.15.96:0/2831500184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7effec0c4ac0 0x7effec0c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 --2- 172.21.15.96:0/2831500184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec06e2c0 0x7effec0c4580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 -- 172.21.15.96:0/2831500184 >> 172.21.15.96:0/2831500184 conn(0x7effec06eea0 msgr2=0x7effec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 -- 172.21.15.96:0/2831500184 shutdown_connections 2024-09-21T02:12:35.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.255+0000 7efff166f640 1 -- 172.21.15.96:0/2831500184 wait complete. 2024-09-21T02:12:35.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 Processor -- start 2024-09-21T02:12:35.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 -- start start 2024-09-21T02:12:35.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7effec06e2c0 0x7effec1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec0c4ac0 0x7effec1e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 0x7effec1e7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effec0c5410 con 0x7effec0cdbf0 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effec0c5590 con 0x7effec0c4ac0 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7effec0ce710 con 0x7effec06e2c0 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 0x7effec1e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 0x7effec1e7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43738/0 (socket says 172.21.15.96:43738) 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 -- 172.21.15.96:0/1057713093 learned_addr learned my addr 172.21.15.96:0/1057713093 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effea7fc640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec0c4ac0 0x7effec1e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeaffd640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7effec06e2c0 0x7effec1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 -- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7effec06e2c0 msgr2=0x7effec1f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7effec06e2c0 0x7effec1f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 -- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec0c4ac0 msgr2=0x7effec1e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec0c4ac0 0x7effec1e7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 -- 172.21.15.96:0/1057713093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec15e060 con 0x7effec0cdbf0 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effeb7fe640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 0x7effec1e7790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effdc00fa70 tx=0x7effdc00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc01a070 con 0x7effec0cdbf0 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effdc008590 con 0x7effec0cdbf0 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec157950 con 0x7effec0cdbf0 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.259+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc006980 con 0x7effec0cdbf0 2024-09-21T02:12:35.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec15e5e0 con 0x7effec0cdbf0 2024-09-21T02:12:35.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effec0c4580 con 0x7effec0cdbf0 2024-09-21T02:12:35.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7effdc01d070 con 0x7effec0cdbf0 2024-09-21T02:12:35.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7effe37fe640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7effcc073200 0x7effcc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7effdc0968b0 con 0x7effec0cdbf0 2024-09-21T02:12:35.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7effeaffd640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7effcc073200 0x7effcc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.263+0000 7effeaffd640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7effcc073200 0x7effcc075710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effe40af000 tx=0x7effe401b1e0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.267+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7effdc061b60 con 0x7effec0cdbf0 2024-09-21T02:12:35.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:35.496 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:35.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.511+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7effec073310 con 0x7effec0cdbf0 2024-09-21T02:12:35.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.687+0000 7effe37fe640 1 -- 172.21.15.96:0/1057713093 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v160) ==== 120+0+0 (secure 0 0 0) 0x7effdc061460 con 0x7effec0cdbf0 2024-09-21T02:12:35.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7effcc073200 msgr2=0x7effcc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7effcc073200 0x7effcc075710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effe40af000 tx=0x7effe401b1e0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 msgr2=0x7effec1e7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 0x7effec1e7790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effdc00fa70 tx=0x7effdc00ff30 comp rx=0 tx=0).stop 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 shutdown_connections 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7effcc073200 0x7effcc075710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effec0cdbf0 0x7effec1e7790 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec0c4ac0 0x7effec1e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 --2- 172.21.15.96:0/1057713093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7effec06e2c0 0x7effec1f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 >> 172.21.15.96:0/1057713093 conn(0x7effec06eea0 msgr2=0x7effec159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 shutdown_connections 2024-09-21T02:12:35.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:35.691+0000 7efff166f640 1 -- 172.21.15.96:0/1057713093 wait complete. 2024-09-21T02:12:35.720 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-09-21T02:12:35.721 INFO:tasks.cephfs_test_runner: 2024-09-21T02:12:35.721 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T02:12:35.721 INFO:tasks.cephfs_test_runner:Ran 1 test in 150.630s 2024-09-21T02:12:35.721 INFO:tasks.cephfs_test_runner: 2024-09-21T02:12:35.721 INFO:tasks.cephfs_test_runner:OK 2024-09-21T02:12:35.721 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T02:12:35.730 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-21T02:12:35.740 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-21T02:12:35.740 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:12:35.740 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:35.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20690 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:12:35.746 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-21T02:12:35.746 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:35.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20690 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:12:35.791 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:12:35.791 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:35.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336312k,nr_inodes=4084078,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16384 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274648k,nr_inodes=818662,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:12:35.797 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T02:12:35.797 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:12:35.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336312k,nr_inodes=4084078,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16384 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274648k,nr_inodes=818662,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274652k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:12:35.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:35.843 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-21T02:12:35.891 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:12:35.891 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:12:35.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:35.891 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:12:35.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:35.948 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:12:36.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:36.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-21T02:12:36.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:36.212 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-21T02:12:36.218 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:12:36.218 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:12:36.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:36.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:12:36.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:36.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:12:36.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:12:36.364 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-09-21T02:12:36.492 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T02:12:36.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:12:36.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/1617764124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e68004d50 msgr2=0x7f1e68005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/1617764124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e68004d50 0x7f1e68005200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c290 tx=0x7f1e7801f6e0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e87fff640 1 -- 172.21.15.96:0/1617764124 reap_dead start 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/1617764124 shutdown_connections 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/1617764124 >> 172.21.15.96:0/1617764124 conn(0x7f1e8806eea0 msgr2=0x7f1e8806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/1617764124 shutdown_connections 2024-09-21T02:12:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.659+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/1617764124 wait complete. 2024-09-21T02:12:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 Processor -- start 2024-09-21T02:12:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 -- start start 2024-09-21T02:12:36.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68004d50 0x7f1e881f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 0x7f1e881e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e881587e0 0x7f1e881e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8815cbb0 con 0x7f1e8814b760 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8815ca30 con 0x7f1e68004d50 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e8dbc0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1e880c4580 con 0x7f1e881587e0 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 0x7f1e881e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 0x7f1e881e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43756/0 (socket says 172.21.15.96:43756) 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 -- 172.21.15.96:0/3949065976 learned_addr learned my addr 172.21.15.96:0/3949065976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e87fff640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e881587e0 0x7f1e881e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e877fe640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68004d50 0x7f1e881f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 -- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e881587e0 msgr2=0x7f1e881e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e881587e0 0x7f1e881e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 -- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68004d50 msgr2=0x7f1e881f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68004d50 0x7f1e881f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 -- 172.21.15.96:0/3949065976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880ca150 con 0x7f1e8814b760 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e86ffd640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 0x7f1e881e6760 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000d550 tx=0x7f1e7000da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70016020 con 0x7f1e8814b760 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e70012630 con 0x7f1e8814b760 2024-09-21T02:12:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.663+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70012a40 con 0x7f1e8814b760 2024-09-21T02:12:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e881e5a60 con 0x7f1e8814b760 2024-09-21T02:12:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e881f3320 con 0x7f1e8814b760 2024-09-21T02:12:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e880cf010 con 0x7f1e8814b760 2024-09-21T02:12:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e70019020 con 0x7f1e8814b760 2024-09-21T02:12:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e7ffff640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1e6c073230 0x7f1e6c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f1e70093800 con 0x7f1e8814b760 2024-09-21T02:12:36.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e877fe640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1e6c073230 0x7f1e6c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.667+0000 7f1e877fe640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1e6c073230 0x7f1e6c075740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009fc0 tx=0x7f1e80009a80 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.671+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1e70020070 con 0x7f1e8814b760 2024-09-21T02:12:36.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.843+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e880ca430 con 0x7f1e6c073230 2024-09-21T02:12:36.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e7ffff640 1 -- 172.21.15.96:0/3949065976 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13477 (secure 0 0 0) 0x7f1e880ca430 con 0x7f1e6c073230 2024-09-21T02:12:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:36.852 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:12:36.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1e6c073230 msgr2=0x7f1e6c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1e6c073230 0x7f1e6c075740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009fc0 tx=0x7f1e80009a80 comp rx=0 tx=0).stop 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 msgr2=0x7f1e881e6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 0x7f1e881e6760 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000d550 tx=0x7f1e7000da10 comp rx=0 tx=0).stop 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 shutdown_connections 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1e6c073230 0x7f1e6c075740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e881587e0 0x7f1e881e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8814b760 0x7f1e881e6760 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 --2- 172.21.15.96:0/3949065976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68004d50 0x7f1e881f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.847+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 >> 172.21.15.96:0/3949065976 conn(0x7f1e8806eea0 msgr2=0x7f1e880c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.851+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 shutdown_connections 2024-09-21T02:12:36.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:36.851+0000 7f1e8dbc0640 1 -- 172.21.15.96:0/3949065976 wait complete. 2024-09-21T02:12:36.875 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-21T02:12:36.578836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229740,"kb_used_data":5184,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744980,"statfs":{"total":773094113280,"available":772858859520,"internally_reserved":0,"allocated":5308416,"data_stored":2976368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002342"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":156,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T02:12:33.670193+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:12:33.670193+0000","last_peered":"2024-09-21T02:12:33.670193+0000","last_clean":"2024-09-21T02:12:33.670193+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:12:33.670193+0000","last_undegraded":"2024-09-21T02:12:33.670193+0000","last_fullsized":"2024-09-21T02:12:33.670193+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":820959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":224131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1105920,"data_stored":813853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:12:36.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:12:37.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 -- 172.21.15.96:0/3049632096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 msgr2=0x7f19c0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 --2- 172.21.15.96:0/3049632096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19c0004c20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f19c400bb10 tx=0x7f19c401cda0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 -- 172.21.15.96:0/3049632096 shutdown_connections 2024-09-21T02:12:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 --2- 172.21.15.96:0/3049632096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19c0004c20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 -- 172.21.15.96:0/3049632096 >> 172.21.15.96:0/3049632096 conn(0x7f19d406eea0 msgr2=0x7f19d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 -- 172.21.15.96:0/3049632096 shutdown_connections 2024-09-21T02:12:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.019+0000 7f19dbd92640 1 -- 172.21.15.96:0/3049632096 wait complete. 2024-09-21T02:12:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 Processor -- start 2024-09-21T02:12:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 -- start start 2024-09-21T02:12:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19d415f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19d41510b0 0x7f19d4142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19d4142560 0x7f19d4142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19d415cbb0 con 0x7f19c0004810 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f19d415ca30 con 0x7f19d4142560 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f19d40c50a0 con 0x7f19d41510b0 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19d415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19d415f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57796/0 (socket says 172.21.15.96:57796) 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 -- 172.21.15.96:0/1530725954 learned_addr learned my addr 172.21.15.96:0/1530725954 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19da32f640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19d4142560 0x7f19d4142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d932d640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19d41510b0 0x7f19d4142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 -- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19d41510b0 msgr2=0x7f19d4142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19d41510b0 0x7f19d4142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 -- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19d4142560 msgr2=0x7f19d4142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19d4142560 0x7f19d4142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 -- 172.21.15.96:0/1530725954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d40b4110 con 0x7f19c0004810 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19d9b2e640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19d415f740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f19d000d820 tx=0x7f19d000dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0019070 con 0x7f19c0004810 2024-09-21T02:12:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d00052e0 con 0x7f19c0004810 2024-09-21T02:12:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d40b43f0 con 0x7f19c0004810 2024-09-21T02:12:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d000b8c0 con 0x7f19c0004810 2024-09-21T02:12:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.023+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d41416a0 con 0x7f19c0004810 2024-09-21T02:12:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.027+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d40ce400 con 0x7f19c0004810 2024-09-21T02:12:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.027+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19d001c070 con 0x7f19c0004810 2024-09-21T02:12:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.027+0000 7f19ca7fc640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b8073200 0x7f19b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.027+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f19d0093940 con 0x7f19c0004810 2024-09-21T02:12:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.027+0000 7f19d932d640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b8073200 0x7f19b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.027+0000 7f19d932d640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b8073200 0x7f19b8075710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b0000 tx=0x7f19c4019b40 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:37.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.031+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19d005eb60 con 0x7f19c0004810 2024-09-21T02:12:37.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.187+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19d415fc80 con 0x7f19b8073200 2024-09-21T02:12:37.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19ca7fc640 1 -- 172.21.15.96:0/1530725954 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13477 (secure 0 0 0) 0x7f19d415fc80 con 0x7f19b8073200 2024-09-21T02:12:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:37.195 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b8073200 msgr2=0x7f19b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b8073200 0x7f19b8075710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b0000 tx=0x7f19c4019b40 comp rx=0 tx=0).stop 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 msgr2=0x7f19d415f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19d415f740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f19d000d820 tx=0x7f19d000dce0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 shutdown_connections 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f19b8073200 0x7f19b8075710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19d4142560 0x7f19d4142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f19d41510b0 0x7f19d4142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 --2- 172.21.15.96:0/1530725954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0004810 0x7f19d415f740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 >> 172.21.15.96:0/1530725954 conn(0x7f19d406eea0 msgr2=0x7f19d40c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 shutdown_connections 2024-09-21T02:12:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.191+0000 7f19dbd92640 1 -- 172.21.15.96:0/1530725954 wait complete. 2024-09-21T02:12:37.217 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-21T02:12:36.578836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229740,"kb_used_data":5184,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744980,"statfs":{"total":773094113280,"available":772858859520,"internally_reserved":0,"allocated":5308416,"data_stored":2976368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002342"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":156,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T02:12:33.670193+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:12:33.670193+0000","last_peered":"2024-09-21T02:12:33.670193+0000","last_clean":"2024-09-21T02:12:33.670193+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:12:33.670193+0000","last_undegraded":"2024-09-21T02:12:33.670193+0000","last_fullsized":"2024-09-21T02:12:33.670193+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":820959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":224131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1105920,"data_stored":813853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:12:37.218 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:12:37.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:12:37.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 -- 172.21.15.96:0/3073939503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15f750 msgr2=0x7fe1bc0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/3073939503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15f750 0x7fe1bc0cff20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0cb5f0 tx=0x7fe1ac01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 -- 172.21.15.96:0/3073939503 shutdown_connections 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/3073939503 >> v2:172.21.15.122:3300/0 conn(0x7fe1bc15fb60 0x7fe1bc0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/3073939503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15f750 0x7fe1bc0cff20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 -- 172.21.15.96:0/3073939503 >> 172.21.15.96:0/3073939503 conn(0x7fe1bc06fbe0 msgr2=0x7fe1bc06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 -- 172.21.15.96:0/3073939503 shutdown_connections 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.359+0000 7fe1c19d9640 1 -- 172.21.15.96:0/3073939503 wait complete. 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 Processor -- start 2024-09-21T02:12:37.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- start start 2024-09-21T02:12:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1bc15b310 0x7fe1bc15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1bc15f750 0x7fe1bc154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 0x7fe1bc1f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc168680 con 0x7fe1bc15fb60 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc168500 con 0x7fe1bc15f750 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe1bc168800 con 0x7fe1bc15b310 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 0x7fe1bc1f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 0x7fe1bc1f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57818/0 (socket says 172.21.15.96:57818) 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/4281949046 learned_addr learned my addr 172.21.15.96:0/4281949046 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1ba7fc640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1bc15f750 0x7fe1bc154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1baffd640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1bc15b310 0x7fe1bc15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1bc15b310 msgr2=0x7fe1bc15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1bc15b310 0x7fe1bc15b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1bc15f750 msgr2=0x7fe1bc154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1bc15f750 0x7fe1bc154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/4281949046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc15e090 con 0x7fe1bc15fb60 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1bb7fe640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 0x7fe1bc1f88f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4007c30 tx=0x7fe1b400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b401a070 con 0x7fe1bc15fb60 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b4007e40 con 0x7fe1bc15fb60 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc15e370 con 0x7fe1bc15fb60 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400ed90 con 0x7fe1bc15fb60 2024-09-21T02:12:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc15e930 con 0x7fe1bc15fb60 2024-09-21T02:12:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.363+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc15bfa0 con 0x7fe1bc15fb60 2024-09-21T02:12:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.367+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe1b401d070 con 0x7fe1bc15fb60 2024-09-21T02:12:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.367+0000 7fe1a3fff640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe1a8073230 0x7fe1a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.367+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fe1b4095b90 con 0x7fe1bc15fb60 2024-09-21T02:12:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.367+0000 7fe1baffd640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe1a8073230 0x7fe1a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.367+0000 7fe1baffd640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe1a8073230 0x7fe1a8075740 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000a750 tx=0x7fe1b0008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.371+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe1b4024080 con 0x7fe1bc15fb60 2024-09-21T02:12:37.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1bc15eca0 con 0x7fe1a8073230 2024-09-21T02:12:37.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1a3fff640 1 -- 172.21.15.96:0/4281949046 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13477 (secure 0 0 0) 0x7fe1bc15eca0 con 0x7fe1a8073230 2024-09-21T02:12:37.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:37.544 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:12:37.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe1a8073230 msgr2=0x7fe1a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe1a8073230 0x7fe1a8075740 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000a750 tx=0x7fe1b0008040 comp rx=0 tx=0).stop 2024-09-21T02:12:37.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 msgr2=0x7fe1bc1f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 0x7fe1bc1f88f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4007c30 tx=0x7fe1b400e420 comp rx=0 tx=0).stop 2024-09-21T02:12:37.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 shutdown_connections 2024-09-21T02:12:37.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe1a8073230 0x7fe1a8075740 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1bc15fb60 0x7fe1bc1f88f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1bc15f750 0x7fe1bc154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 --2- 172.21.15.96:0/4281949046 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1bc15b310 0x7fe1bc15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 >> 172.21.15.96:0/4281949046 conn(0x7fe1bc06fbe0 msgr2=0x7fe1bc073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:37.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 shutdown_connections 2024-09-21T02:12:37.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.539+0000 7fe1c19d9640 1 -- 172.21.15.96:0/4281949046 wait complete. 2024-09-21T02:12:37.565 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-21T02:12:36.578836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229740,"kb_used_data":5184,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744980,"statfs":{"total":773094113280,"available":772858859520,"internally_reserved":0,"allocated":5308416,"data_stored":2976368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002342"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":156,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T02:12:33.670193+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:12:33.670193+0000","last_peered":"2024-09-21T02:12:33.670193+0000","last_clean":"2024-09-21T02:12:33.670193+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:12:33.670193+0000","last_undegraded":"2024-09-21T02:12:33.670193+0000","last_fullsized":"2024-09-21T02:12:33.670193+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":820959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":224131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":223485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1105920,"data_stored":813853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:12:37.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2273976088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d40cc630 msgr2=0x7f55d406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2273976088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d40cc630 0x7f55d406d410 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f55d4075350 tx=0x7f55c401c830 comp rx=0 tx=0).stop 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2273976088 shutdown_connections 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2273976088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d40cc630 0x7f55d406d410 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2273976088 >> v2:172.21.15.122:3300/0 conn(0x7f55d4071920 0x7f55d40cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2273976088 >> 172.21.15.96:0/2273976088 conn(0x7f55d406fbe0 msgr2=0x7f55d406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2273976088 shutdown_connections 2024-09-21T02:12:37.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.699+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2273976088 wait complete. 2024-09-21T02:12:37.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 Processor -- start 2024-09-21T02:12:37.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 -- start start 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d40cc630 0x7f55d4151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 0x7f55d4151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d415c9b0 0x7f55d40b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55d416bfc0 con 0x7f55d415e5e0 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55d416be40 con 0x7f55d40cc630 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f55d416c140 con 0x7f55d415c9b0 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 0x7f55d4151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 0x7f55d4151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57824/0 (socket says 172.21.15.96:57824) 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 -- 172.21.15.96:0/2369553438 learned_addr learned my addr 172.21.15.96:0/2369553438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d3fff640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d415c9b0 0x7f55d40b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d37fe640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d40cc630 0x7f55d4151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 -- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d415c9b0 msgr2=0x7f55d40b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d415c9b0 0x7f55d40b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 -- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d40cc630 msgr2=0x7f55d4151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d40cc630 0x7f55d4151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 -- 172.21.15.96:0/2369553438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d4153ff0 con 0x7f55d415e5e0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d2ffd640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 0x7f55d4151770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f55c800d720 tx=0x7f55c800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8017070 con 0x7f55d415e5e0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55c8009d20 con 0x7f55d415e5e0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d41542d0 con 0x7f55d415e5e0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8005280 con 0x7f55d415e5e0 2024-09-21T02:12:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.703+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d4143990 con 0x7f55d415e5e0 2024-09-21T02:12:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.707+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d4154510 con 0x7f55d415e5e0 2024-09-21T02:12:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.707+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55c8005420 con 0x7f55d415e5e0 2024-09-21T02:12:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.707+0000 7f55bbfff640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f55c0073200 0x7f55c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.707+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f55c8094150 con 0x7f55d415e5e0 2024-09-21T02:12:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.707+0000 7f55d37fe640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f55c0073200 0x7f55c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.707+0000 7f55d37fe640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f55c0073200 0x7f55c0075710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55cc001cf0 tx=0x7f55cc009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:37.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.711+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f55d4154510 con 0x7f55d415e5e0 2024-09-21T02:12:37.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.871+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55d40cc0f0 con 0x7f55d415e5e0 2024-09-21T02:12:37.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55bbfff640 1 -- 172.21.15.96:0/2369553438 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13305 (secure 0 0 0) 0x7f55c805f370 con 0x7f55d415e5e0 2024-09-21T02:12:37.879 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:37.879 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":45,"fsid":"892bf3fa-0a6f-4242-b0ca-536a4d478630","created":"2024-09-21T02:08:44.887145+0000","modified":"2024-09-21T02:12:34.678665+0000","last_up_change":"2024-09-21T02:08:59.105238+0000","last_in_change":"2024-09-21T02:08:49.514910+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:09:00.544933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e39f309-6001-4136-ae9d-cef25153746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6801","nonce":190673067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6803","nonce":190673067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6807","nonce":190673067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":190673067},{"type":"v1","addr":"172.21.15.96:6805","nonce":190673067}]},"public_addr":"172.21.15.96:6801/190673067","cluster_addr":"172.21.15.96:6803/190673067","heartbeat_back_addr":"172.21.15.96:6807/190673067","heartbeat_front_addr":"172.21.15.96:6805/190673067","state":["exists","up"]},{"osd":1,"uuid":"51c3338d-b835-47ea-8f9d-59786a4bd69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6817","nonce":2322744690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6819","nonce":2322744690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6823","nonce":2322744690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2322744690},{"type":"v1","addr":"172.21.15.96:6821","nonce":2322744690}]},"public_addr":"172.21.15.96:6817/2322744690","cluster_addr":"172.21.15.96:6819/2322744690","heartbeat_back_addr":"172.21.15.96:6823/2322744690","heartbeat_front_addr":"172.21.15.96:6821/2322744690","state":["exists","up"]},{"osd":2,"uuid":"ee46c0cd-1c7a-440c-aaa1-6888f5787b40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6825","nonce":1085819941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6827","nonce":1085819941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6831","nonce":1085819941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1085819941},{"type":"v1","addr":"172.21.15.96:6829","nonce":1085819941}]},"public_addr":"172.21.15.96:6825/1085819941","cluster_addr":"172.21.15.96:6827/1085819941","heartbeat_back_addr":"172.21.15.96:6831/1085819941","heartbeat_front_addr":"172.21.15.96:6829/1085819941","state":["exists","up"]},{"osd":3,"uuid":"4e786efb-1547-443c-85e9-1ac57a64c176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6809","nonce":3409109307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6811","nonce":3409109307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6815","nonce":3409109307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3409109307},{"type":"v1","addr":"172.21.15.96:6813","nonce":3409109307}]},"public_addr":"172.21.15.96:6809/3409109307","cluster_addr":"172.21.15.96:6811/3409109307","heartbeat_back_addr":"172.21.15.96:6815/3409109307","heartbeat_front_addr":"172.21.15.96:6813/3409109307","state":["exists","up"]},{"osd":4,"uuid":"09833021-edc8-4e94-ac1e-e0a54df57434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6803","nonce":567414413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6805","nonce":567414413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6809","nonce":567414413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":567414413},{"type":"v1","addr":"172.21.15.122:6807","nonce":567414413}]},"public_addr":"172.21.15.122:6803/567414413","cluster_addr":"172.21.15.122:6805/567414413","heartbeat_back_addr":"172.21.15.122:6809/567414413","heartbeat_front_addr":"172.21.15.122:6807/567414413","state":["exists","up"]},{"osd":5,"uuid":"562a7d36-d24f-48ea-9dd8-cb8366e22cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6811","nonce":4177593286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6813","nonce":4177593286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6817","nonce":4177593286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4177593286},{"type":"v1","addr":"172.21.15.122:6815","nonce":4177593286}]},"public_addr":"172.21.15.122:6811/4177593286","cluster_addr":"172.21.15.122:6813/4177593286","heartbeat_back_addr":"172.21.15.122:6817/4177593286","heartbeat_front_addr":"172.21.15.122:6815/4177593286","state":["exists","up"]},{"osd":6,"uuid":"984413c0-d6d2-4d1f-8675-a3ad89f2939c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6823","nonce":3298887226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6827","nonce":3298887226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6833","nonce":3298887226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":3298887226},{"type":"v1","addr":"172.21.15.122:6831","nonce":3298887226}]},"public_addr":"172.21.15.122:6823/3298887226","cluster_addr":"172.21.15.122:6827/3298887226","heartbeat_back_addr":"172.21.15.122:6833/3298887226","heartbeat_front_addr":"172.21.15.122:6831/3298887226","state":["exists","up"]},{"osd":7,"uuid":"fcd138eb-65fc-4ab2-b493-3935f86d7c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6819","nonce":1732471888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6822","nonce":1732471888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6830","nonce":1732471888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1732471888},{"type":"v1","addr":"172.21.15.122:6826","nonce":1732471888}]},"public_addr":"172.21.15.122:6819/1732471888","cluster_addr":"172.21.15.122:6822/1732471888","heartbeat_back_addr":"172.21.15.122:6830/1732471888","heartbeat_front_addr":"172.21.15.122:6826/1732471888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.934017+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.671191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.215565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.073154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:55.989015+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.165703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.598319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:08:56.572765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6835/3851681791":"2024-09-22T02:12:31.593767+0000","172.21.15.96:6834/3851681791":"2024-09-22T02:12:31.593767+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:12:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f55c0073200 msgr2=0x7f55c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f55c0073200 0x7f55c0075710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55cc001cf0 tx=0x7f55cc009040 comp rx=0 tx=0).stop 2024-09-21T02:12:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 msgr2=0x7f55d4151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 0x7f55d4151770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f55c800d720 tx=0x7f55c800dbe0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 shutdown_connections 2024-09-21T02:12:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f55c0073200 0x7f55c0075710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d415c9b0 0x7f55d40b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55d415e5e0 0x7f55d4151770 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 --2- 172.21.15.96:0/2369553438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d40cc630 0x7f55d4151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 >> 172.21.15.96:0/2369553438 conn(0x7f55d406fbe0 msgr2=0x7f55d40b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 shutdown_connections 2024-09-21T02:12:37.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:37.875+0000 7f55d9cf6640 1 -- 172.21.15.96:0/2369553438 wait complete. 2024-09-21T02:12:38.901 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T02:12:38.901 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:39.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 -- 172.21.15.96:0/540615336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 msgr2=0x7f6f300cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 --2- 172.21.15.96:0/540615336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f300cf6d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009870 tx=0x7f6f2001c700 comp rx=0 tx=0).stop 2024-09-21T02:12:39.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 -- 172.21.15.96:0/540615336 shutdown_connections 2024-09-21T02:12:39.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 --2- 172.21.15.96:0/540615336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f300cf6d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 -- 172.21.15.96:0/540615336 >> 172.21.15.96:0/540615336 conn(0x7f6f300d05a0 msgr2=0x7f6f300d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 -- 172.21.15.96:0/540615336 shutdown_connections 2024-09-21T02:12:39.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 -- 172.21.15.96:0/540615336 wait complete. 2024-09-21T02:12:39.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 Processor -- start 2024-09-21T02:12:39.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.063+0000 7f6f35142640 1 -- start start 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6f3006f700 0x7f6f3013c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f3013b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f300cbed0 0x7f6f30159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f30169b00 con 0x7f6f300b2ef0 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f30169980 con 0x7f6f300cbed0 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6f30169c80 con 0x7f6f3006f700 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f3013b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f3013b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57848/0 (socket says 172.21.15.96:57848) 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 -- 172.21.15.96:0/1298433717 learned_addr learned my addr 172.21.15.96:0/1298433717 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2ed9d640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6f3006f700 0x7f6f3013c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2f59e640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f300cbed0 0x7f6f30159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 -- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6f3006f700 msgr2=0x7f6f3013c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6f3006f700 0x7f6f3013c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 -- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f300cbed0 msgr2=0x7f6f30159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f300cbed0 0x7f6f30159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 -- 172.21.15.96:0/1298433717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3013caa0 con 0x7f6f300b2ef0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2f59e640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f300cbed0 0x7f6f30159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f2659c640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f3013b870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800b3c0 tx=0x7f6f1800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18011830 con 0x7f6f300b2ef0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f18016070 con 0x7f6f300b2ef0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3013cd80 con 0x7f6f300b2ef0 2024-09-21T02:12:39.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18011d40 con 0x7f6f300b2ef0 2024-09-21T02:12:39.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3013d340 con 0x7f6f300b2ef0 2024-09-21T02:12:39.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f300cf6d0 con 0x7f6f300b2ef0 2024-09-21T02:12:39.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6f1801b460 con 0x7f6f300b2ef0 2024-09-21T02:12:39.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6f10073200 0x7f6f10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f6f18094830 con 0x7f6f300b2ef0 2024-09-21T02:12:39.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7f6f100796f0 0x7f6f1007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 --> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f1800de50 con 0x7f6f100796f0 2024-09-21T02:12:39.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.067+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6f18094cc0 con 0x7f6f300b2ef0 2024-09-21T02:12:39.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.071+0000 7f6f2ed9d640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6f10073200 0x7f6f10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.071+0000 7f6f2f59e640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7f6f100796f0 0x7f6f1007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.071+0000 7f6f2f59e640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7f6f100796f0 0x7f6f1007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.071+0000 7f6f2ed9d640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6f10073200 0x7f6f10075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f200b2040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.071+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== osd.0 v2:172.21.15.96:6800/190673067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f1800de50 con 0x7f6f100796f0 2024-09-21T02:12:39.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 --> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6f30074a10 con 0x7f6f100796f0 2024-09-21T02:12:39.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f27fff640 1 -- 172.21.15.96:0/1298433717 <== osd.0 v2:172.21.15.96:6800/190673067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6f30074a10 con 0x7f6f100796f0 2024-09-21T02:12:39.089 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:39.089 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:39.089 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7f6f100796f0 msgr2=0x7f6f1007bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:6800/190673067,v1:172.21.15.96:6801/190673067] conn(0x7f6f100796f0 0x7f6f1007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6f10073200 msgr2=0x7f6f10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6f10073200 0x7f6f10075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f200b2040 comp rx=0 tx=0).stop 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 msgr2=0x7f6f3013b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 --2- 172.21.15.96:0/1298433717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f300b2ef0 0x7f6f3013b870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800b3c0 tx=0x7f6f1800b880 comp rx=0 tx=0).stop 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f2f59e640 1 -- 172.21.15.96:0/1298433717 reap_dead start 2024-09-21T02:12:39.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 shutdown_connections 2024-09-21T02:12:39.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 >> 172.21.15.96:0/1298433717 conn(0x7f6f300d05a0 msgr2=0x7f6f3006ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 shutdown_connections 2024-09-21T02:12:39.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.083+0000 7f6f35142640 1 -- 172.21.15.96:0/1298433717 wait complete. 2024-09-21T02:12:39.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T02:12:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16f003a640 1 -- 172.21.15.96:0/3614985870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d4003a10 msgr2=0x7f16d4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16f003a640 1 --2- 172.21.15.96:0/3614985870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d4003a10 0x7f16d4003e80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f16e400bbe0 tx=0x7f16e401ce50 comp rx=0 tx=0).stop 2024-09-21T02:12:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16ee5d7640 1 -- 172.21.15.96:0/3614985870 reap_dead start 2024-09-21T02:12:39.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16f003a640 1 -- 172.21.15.96:0/3614985870 shutdown_connections 2024-09-21T02:12:39.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16f003a640 1 -- 172.21.15.96:0/3614985870 >> 172.21.15.96:0/3614985870 conn(0x7f16e806eea0 msgr2=0x7f16e806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16f003a640 1 -- 172.21.15.96:0/3614985870 shutdown_connections 2024-09-21T02:12:39.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.251+0000 7f16f003a640 1 -- 172.21.15.96:0/3614985870 wait complete. 2024-09-21T02:12:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 Processor -- start 2024-09-21T02:12:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 -- start start 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16d4003a10 0x7f16e80c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16e8154920 0x7f16e815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 0x7f16e8157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8165ce0 con 0x7f16e80c6ec0 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8165b60 con 0x7f16e8154920 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16f003a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f16e8165e60 con 0x7f16d4003a10 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 0x7f16e8157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 0x7f16e8157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57868/0 (socket says 172.21.15.96:57868) 2024-09-21T02:12:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 -- 172.21.15.96:0/1575837223 learned_addr learned my addr 172.21.15.96:0/1575837223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16eddd6640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16d4003a10 0x7f16e80c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ed5d5640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16e8154920 0x7f16e815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 -- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16d4003a10 msgr2=0x7f16e80c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16d4003a10 0x7f16e80c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 -- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16e8154920 msgr2=0x7f16e815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16e8154920 0x7f16e815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 -- 172.21.15.96:0/1575837223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e8154d30 con 0x7f16e80c6ec0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ed5d5640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16e8154920 0x7f16e815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.255+0000 7f16ee5d7640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 0x7f16e8157390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f16e80cdba0 tx=0x7f16e000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0017070 con 0x7f16e80c6ec0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16e000a860 con 0x7f16e80c6ec0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e000acb0 con 0x7f16e80c6ec0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e8156700 con 0x7f16e80c6ec0 2024-09-21T02:12:39.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e8159ae0 con 0x7f16e80c6ec0 2024-09-21T02:12:39.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e806e9d0 con 0x7f16e80c6ec0 2024-09-21T02:12:39.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f16e001a070 con 0x7f16e80c6ec0 2024-09-21T02:12:39.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16de7fc640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f16cc073200 0x7f16cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f16e0093640 con 0x7f16e80c6ec0 2024-09-21T02:12:39.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16eddd6640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f16cc073200 0x7f16cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.259+0000 7f16eddd6640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f16cc073200 0x7f16cc075710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f16e40b0000 tx=0x7f16e4019bf0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.263+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f16e806e7f0 con 0x7f16e80c6ec0 2024-09-21T02:12:39.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.439+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f16e8156240 con 0x7f16cc073200 2024-09-21T02:12:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.439+0000 7f16de7fc640 1 -- 172.21.15.96:0/1575837223 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f16e4005020 con 0x7f16cc073200 2024-09-21T02:12:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.439+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f16cc073200 msgr2=0x7f16cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f16cc073200 0x7f16cc075710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f16e40b0000 tx=0x7f16e4019bf0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 msgr2=0x7f16e8157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 0x7f16e8157390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f16e80cdba0 tx=0x7f16e000cbd0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 shutdown_connections 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f16cc073200 0x7f16cc075710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16e80c6ec0 0x7f16e8157390 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16e8154920 0x7f16e815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 --2- 172.21.15.96:0/1575837223 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f16d4003a10 0x7f16e80c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 >> 172.21.15.96:0/1575837223 conn(0x7f16e806eea0 msgr2=0x7f16e80cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 shutdown_connections 2024-09-21T02:12:39.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.443+0000 7f16f003a640 1 -- 172.21.15.96:0/1575837223 wait complete. 2024-09-21T02:12:39.467 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T02:12:39.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa93246a640 1 -- 172.21.15.96:0/3946240407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa914003a10 msgr2=0x7fa914003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa93246a640 1 --2- 172.21.15.96:0/3946240407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa914003a10 0x7fa914003e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009b40 tx=0x7fa91c01d290 comp rx=0 tx=0).stop 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa930a07640 1 -- 172.21.15.96:0/3946240407 reap_dead start 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa93246a640 1 -- 172.21.15.96:0/3946240407 shutdown_connections 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa93246a640 1 -- 172.21.15.96:0/3946240407 >> 172.21.15.96:0/3946240407 conn(0x7fa92c0d42b0 msgr2=0x7fa92c0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa93246a640 1 -- 172.21.15.96:0/3946240407 shutdown_connections 2024-09-21T02:12:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.627+0000 7fa93246a640 1 -- 172.21.15.96:0/3946240407 wait complete. 2024-09-21T02:12:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 Processor -- start 2024-09-21T02:12:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- start start 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914003a10 0x7fa92c168840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa92c15fd40 0x7fa92c1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa92c1608f0 0x7fa92c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0b9010 con 0x7fa92c1608f0 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0b8e90 con 0x7fa914003a10 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c0b9190 con 0x7fa92c15fd40 2024-09-21T02:12:39.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa92c1608f0 0x7fa92c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa92c1608f0 0x7fa92c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57878/0 (socket says 172.21.15.96:57878) 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 -- 172.21.15.96:0/4130637753 learned_addr learned my addr 172.21.15.96:0/4130637753 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa92b7fe640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa92c15fd40 0x7fa92c1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa92bfff640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914003a10 0x7fa92c168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 -- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa92c15fd40 msgr2=0x7fa92c1562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa92c15fd40 0x7fa92c1562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 -- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914003a10 msgr2=0x7fa92c168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914003a10 0x7fa92c168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 -- 172.21.15.96:0/4130637753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c161240 con 0x7fa92c1608f0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa92bfff640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914003a10 0x7fa92c168840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa930a07640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa92c1608f0 0x7fa92c160d00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa924002240 tx=0x7fa92400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924015070 con 0x7fa92c1608f0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa92400ca20 con 0x7fa92c1608f0 2024-09-21T02:12:39.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c15ef80 con 0x7fa92c1608f0 2024-09-21T02:12:39.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92400ce70 con 0x7fa92c1608f0 2024-09-21T02:12:39.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c15f540 con 0x7fa92c1608f0 2024-09-21T02:12:39.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.631+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa92c0b2c40 con 0x7fa92c1608f0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa924018070 con 0x7fa92c1608f0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa918073200 0x7fa918075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fa924094920 con 0x7fa92c1608f0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fa9180796f0 0x7fa91807bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 --> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa924024e50 con 0x7fa9180796f0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa924094db0 con 0x7fa92c1608f0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa92bfff640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa918073200 0x7fa918075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa92b7fe640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fa9180796f0 0x7fa91807bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa92b7fe640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fa9180796f0 0x7fa91807bb80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa92bfff640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa918073200 0x7fa918075710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009b40 tx=0x7fa92c15ed70 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.635+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== osd.1 v2:172.21.15.96:6816/2322744690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa924024e50 con 0x7fa9180796f0 2024-09-21T02:12:39.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.647+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 --> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa92c1690b0 con 0x7fa9180796f0 2024-09-21T02:12:39.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.647+0000 7fa928ff9640 1 -- 172.21.15.96:0/4130637753 <== osd.1 v2:172.21.15.96:6816/2322744690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa92c1690b0 con 0x7fa9180796f0 2024-09-21T02:12:39.654 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:39.654 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:39.655 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:39.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fa9180796f0 msgr2=0x7fa91807bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:6816/2322744690,v1:172.21.15.96:6817/2322744690] conn(0x7fa9180796f0 0x7fa91807bb80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa918073200 msgr2=0x7fa918075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fa918073200 0x7fa918075710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009b40 tx=0x7fa92c15ed70 comp rx=0 tx=0).stop 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa92c1608f0 msgr2=0x7fa92c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 --2- 172.21.15.96:0/4130637753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa92c1608f0 0x7fa92c160d00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa924002240 tx=0x7fa92400ecf0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa930a07640 1 -- 172.21.15.96:0/4130637753 reap_dead start 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 shutdown_connections 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 >> 172.21.15.96:0/4130637753 conn(0x7fa92c0d42b0 msgr2=0x7fa92c15cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 shutdown_connections 2024-09-21T02:12:39.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.651+0000 7fa93246a640 1 -- 172.21.15.96:0/4130637753 wait complete. 2024-09-21T02:12:39.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T02:12:39.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 -- 172.21.15.96:0/1253160987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 msgr2=0x7efef40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1253160987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef40cf010 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efef406a1c0 tx=0x7efeec01ccf0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 -- 172.21.15.96:0/1253160987 shutdown_connections 2024-09-21T02:12:39.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1253160987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef40cf010 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1253160987 >> v2:172.21.15.122:3300/0 conn(0x7efef406e2c0 0x7efef40c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 -- 172.21.15.96:0/1253160987 >> 172.21.15.96:0/1253160987 conn(0x7efef406eea0 msgr2=0x7efef406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 -- 172.21.15.96:0/1253160987 shutdown_connections 2024-09-21T02:12:39.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.815+0000 7efefbdc8640 1 -- 172.21.15.96:0/1253160987 wait complete. 2024-09-21T02:12:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 Processor -- start 2024-09-21T02:12:39.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 -- start start 2024-09-21T02:12:39.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef406e2c0 0x7efef40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef40c4ac0 0x7efef4158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef41590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efef4169c60 con 0x7efef40cec00 2024-09-21T02:12:39.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efef4169ae0 con 0x7efef40c4ac0 2024-09-21T02:12:39.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efef4169de0 con 0x7efef406e2c0 2024-09-21T02:12:39.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef41590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef41590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57892/0 (socket says 172.21.15.96:57892) 2024-09-21T02:12:39.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 -- 172.21.15.96:0/1644771332 learned_addr learned my addr 172.21.15.96:0/1644771332 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:39.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efef9363640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef40c4ac0 0x7efef4158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efef9b64640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef406e2c0 0x7efef40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 -- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef406e2c0 msgr2=0x7efef40b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef406e2c0 0x7efef40b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 -- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef40c4ac0 msgr2=0x7efef4158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef40c4ac0 0x7efef4158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 -- 172.21.15.96:0/1644771332 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef414d4b0 con 0x7efef40cec00 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efef9b64640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef406e2c0 0x7efef40b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefa365640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef41590b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efeec01d1d0 tx=0x7efeec005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0b2040 con 0x7efef40cec00 2024-09-21T02:12:39.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeec0bb070 con 0x7efef40cec00 2024-09-21T02:12:39.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef40b3b80 con 0x7efef40cec00 2024-09-21T02:12:39.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0a5e60 con 0x7efef40cec00 2024-09-21T02:12:39.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.819+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef40b4140 con 0x7efef40cec00 2024-09-21T02:12:39.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.823+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef414d680 con 0x7efef40cec00 2024-09-21T02:12:39.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.823+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efeec004550 con 0x7efef40cec00 2024-09-21T02:12:39.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.823+0000 7efeea7fc640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efed8073200 0x7efed8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.823+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7efeec12f450 con 0x7efef40cec00 2024-09-21T02:12:39.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.823+0000 7efef9b64640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efed8073200 0x7efed8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.823+0000 7efef9b64640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efed8073200 0x7efed8075710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efef0005fd0 tx=0x7efef0005dc0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.827+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efeec016970 con 0x7efef40cec00 2024-09-21T02:12:40.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.999+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7efef4157590 con 0x7efed8073200 2024-09-21T02:12:40.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:39.999+0000 7efeea7fc640 1 -- 172.21.15.96:0/1644771332 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efef4157590 con 0x7efed8073200 2024-09-21T02:12:40.006 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T02:12:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efed8073200 msgr2=0x7efed8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efed8073200 0x7efed8075710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efef0005fd0 tx=0x7efef0005dc0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 msgr2=0x7efef41590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef41590b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efeec01d1d0 tx=0x7efeec005c90 comp rx=0 tx=0).stop 2024-09-21T02:12:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 shutdown_connections 2024-09-21T02:12:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efed8073200 0x7efed8075710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef40cec00 0x7efef41590b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efef40c4ac0 0x7efef4158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 --2- 172.21.15.96:0/1644771332 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efef406e2c0 0x7efef40b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 >> 172.21.15.96:0/1644771332 conn(0x7efef406eea0 msgr2=0x7efef415abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 shutdown_connections 2024-09-21T02:12:40.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.003+0000 7efefbdc8640 1 -- 172.21.15.96:0/1644771332 wait complete. 2024-09-21T02:12:40.030 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T02:12:40.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:40.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 -- 172.21.15.96:0/1424885414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb00731c0 msgr2=0x7fddb00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 --2- 172.21.15.96:0/1424885414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb00731c0 0x7fddb00735d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fddb0070ea0 tx=0x7fdda801c670 comp rx=0 tx=0).stop 2024-09-21T02:12:40.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 -- 172.21.15.96:0/1424885414 shutdown_connections 2024-09-21T02:12:40.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 --2- 172.21.15.96:0/1424885414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb00731c0 0x7fddb00735d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 --2- 172.21.15.96:0/1424885414 >> v2:172.21.15.122:3300/0 conn(0x7fddb015f060 0x7fddb015f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 -- 172.21.15.96:0/1424885414 >> 172.21.15.96:0/1424885414 conn(0x7fddb00d05a0 msgr2=0x7fddb00d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 -- 172.21.15.96:0/1424885414 shutdown_connections 2024-09-21T02:12:40.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.187+0000 7fddb70ab640 1 -- 172.21.15.96:0/1424885414 wait complete. 2024-09-21T02:12:40.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 Processor -- start 2024-09-21T02:12:40.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 -- start start 2024-09-21T02:12:40.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddb00731c0 0x7fddb01fc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb01523d0 0x7fddb01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddb015f060 0x7fddb0148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0168b20 con 0x7fddb01523d0 2024-09-21T02:12:40.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01689a0 con 0x7fddb00731c0 2024-09-21T02:12:40.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb70ab640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fddb0168ca0 con 0x7fddb015f060 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb01523d0 0x7fddb01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb01523d0 0x7fddb01587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57912/0 (socket says 172.21.15.96:57912) 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 -- 172.21.15.96:0/3871572379 learned_addr learned my addr 172.21.15.96:0/3871572379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb4e47640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddb00731c0 0x7fddb01fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddb5648640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddb015f060 0x7fddb0148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 -- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddb015f060 msgr2=0x7fddb0148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddb015f060 0x7fddb0148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 -- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddb00731c0 msgr2=0x7fddb01fc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddb00731c0 0x7fddb01fc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 -- 172.21.15.96:0/3871572379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb014ae80 con 0x7fddb01523d0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddaffff640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb01523d0 0x7fddb01587e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800b3c0 tx=0x7fdd9800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.191+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011830 con 0x7fddb01523d0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd98016070 con 0x7fddb01523d0 2024-09-21T02:12:40.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011d60 con 0x7fddb01523d0 2024-09-21T02:12:40.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb014b160 con 0x7fddb01523d0 2024-09-21T02:12:40.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb014b720 con 0x7fddb01523d0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddb00735d0 con 0x7fddb01523d0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdd9801b460 con 0x7fddb01523d0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdd90073200 0x7fdd90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fdd980946d0 con 0x7fddb01523d0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fdd900796f0 0x7fdd9007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 --> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd9007c250 con 0x7fdd900796f0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdd98094b60 con 0x7fddb01523d0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb4e47640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdd90073200 0x7fdd90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb5648640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fdd900796f0 0x7fdd9007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb5648640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fdd900796f0 0x7fdd9007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddb4e47640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdd90073200 0x7fdd90075710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009fd0 tx=0x7fdda000d040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.195+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== osd.2 v2:172.21.15.96:6824/1085819941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd9007c250 con 0x7fdd900796f0 2024-09-21T02:12:40.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 --> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fddb014ac30 con 0x7fdd900796f0 2024-09-21T02:12:40.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddad7fa640 1 -- 172.21.15.96:0/3871572379 <== osd.2 v2:172.21.15.96:6824/1085819941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fddb014ac30 con 0x7fdd900796f0 2024-09-21T02:12:40.216 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:40.216 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:40.216 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:40.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fdd900796f0 msgr2=0x7fdd9007bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:6824/1085819941,v1:172.21.15.96:6825/1085819941] conn(0x7fdd900796f0 0x7fdd9007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdd90073200 msgr2=0x7fdd90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fdd90073200 0x7fdd90075710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009fd0 tx=0x7fdda000d040 comp rx=0 tx=0).stop 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb01523d0 msgr2=0x7fddb01587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 --2- 172.21.15.96:0/3871572379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb01523d0 0x7fddb01587e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800b3c0 tx=0x7fdd9800b880 comp rx=0 tx=0).stop 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb5648640 1 -- 172.21.15.96:0/3871572379 reap_dead start 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 shutdown_connections 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 >> 172.21.15.96:0/3871572379 conn(0x7fddb00d05a0 msgr2=0x7fddb014c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 shutdown_connections 2024-09-21T02:12:40.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.211+0000 7fddb70ab640 1 -- 172.21.15.96:0/3871572379 wait complete. 2024-09-21T02:12:40.235 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T02:12:40.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 -- 172.21.15.96:0/2068538271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 msgr2=0x7f457c06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 --2- 172.21.15.96:0/2068538271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c06e4b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f456c009bc0 tx=0x7f456c01cb60 comp rx=0 tx=0).stop 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 -- 172.21.15.96:0/2068538271 shutdown_connections 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 --2- 172.21.15.96:0/2068538271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c06e4b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 --2- 172.21.15.96:0/2068538271 >> v2:172.21.15.122:3300/0 conn(0x7f457c0b66a0 0x7f457c0b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 -- 172.21.15.96:0/2068538271 >> 172.21.15.96:0/2068538271 conn(0x7f457c0cda20 msgr2=0x7f457c0cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 -- 172.21.15.96:0/2068538271 shutdown_connections 2024-09-21T02:12:40.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.379+0000 7f45815d4640 1 -- 172.21.15.96:0/2068538271 wait complete. 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 Processor -- start 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 -- start start 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f457c073c70 0x7f457c150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c0b66a0 0x7f457c0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1692b0 con 0x7f457c06e0a0 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0cd4c0 con 0x7f457c0b66a0 2024-09-21T02:12:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f457c169430 con 0x7f457c073c70 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c15b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57928/0 (socket says 172.21.15.96:57928) 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 -- 172.21.15.96:0/329074919 learned_addr learned my addr 172.21.15.96:0/329074919 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457b7fe640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c0b66a0 0x7f457c0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457a7fc640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f457c073c70 0x7f457c150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 -- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f457c073c70 msgr2=0x7f457c150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f457c073c70 0x7f457c150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 -- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c0b66a0 msgr2=0x7f457c0ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c0b66a0 0x7f457c0ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 -- 172.21.15.96:0/329074919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c150fd0 con 0x7f457c06e0a0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f457affd640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c15b300 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45740058d0 tx=0x7f457400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574017070 con 0x7f457c06e0a0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4574009d20 con 0x7f457c06e0a0 2024-09-21T02:12:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c1585a0 con 0x7f457c06e0a0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740106c0 con 0x7f457c06e0a0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.383+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c156780 con 0x7f457c06e0a0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.387+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c074f30 con 0x7f457c06e0a0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.387+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f457401a070 con 0x7f457c06e0a0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.387+0000 7f45737fe640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f455c073200 0x7f455c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.387+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f4574094180 con 0x7f457c06e0a0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.387+0000 7f457a7fc640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f455c073200 0x7f455c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.387+0000 7f457a7fc640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f455c073200 0x7f455c075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f457c1fa3b0 tx=0x7f4564009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.391+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f457405ed50 con 0x7f457c06e0a0 2024-09-21T02:12:40.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.555+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f457c074220 con 0x7f455c073200 2024-09-21T02:12:40.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.555+0000 7f45737fe640 1 -- 172.21.15.96:0/329074919 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f457c074220 con 0x7f455c073200 2024-09-21T02:12:40.561 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f455c073200 msgr2=0x7f455c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f455c073200 0x7f455c075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f457c1fa3b0 tx=0x7f4564009040 comp rx=0 tx=0).stop 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 msgr2=0x7f457c15b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c15b300 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45740058d0 tx=0x7f457400dde0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 shutdown_connections 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f455c073200 0x7f455c075710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c0b66a0 0x7f457c0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f457c073c70 0x7f457c150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 --2- 172.21.15.96:0/329074919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f457c06e0a0 0x7f457c15b300 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 >> 172.21.15.96:0/329074919 conn(0x7f457c0cda20 msgr2=0x7f457c06d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 shutdown_connections 2024-09-21T02:12:40.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.559+0000 7f45815d4640 1 -- 172.21.15.96:0/329074919 wait complete. 2024-09-21T02:12:40.583 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T02:12:40.583 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 -- 172.21.15.96:0/2906919284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c0b3180 msgr2=0x7fcc7c0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 --2- 172.21.15.96:0/2906919284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c0b3180 0x7fcc7c0b3590 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c069ad0 tx=0x7fcc6c01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 -- 172.21.15.96:0/2906919284 shutdown_connections 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 --2- 172.21.15.96:0/2906919284 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc7c06da50 0x7fcc7c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 --2- 172.21.15.96:0/2906919284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c0b3180 0x7fcc7c0b3590 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 --2- 172.21.15.96:0/2906919284 >> v2:172.21.15.122:3300/0 conn(0x7fcc7c15e750 0x7fcc7c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 -- 172.21.15.96:0/2906919284 >> 172.21.15.96:0/2906919284 conn(0x7fcc7c0d42b0 msgr2=0x7fcc7c0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 -- 172.21.15.96:0/2906919284 shutdown_connections 2024-09-21T02:12:40.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 -- 172.21.15.96:0/2906919284 wait complete. 2024-09-21T02:12:40.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 Processor -- start 2024-09-21T02:12:40.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.731+0000 7fcc8394c640 1 -- start start 2024-09-21T02:12:40.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c06da50 0x7fcc7c14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0b3180 0x7fcc7c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc7c15e750 0x7fcc7c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c0cfeb0 con 0x7fcc7c06da50 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c0cfd30 con 0x7fcc7c0b3180 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c0b8170 con 0x7fcc7c15e750 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c06da50 0x7fcc7c14bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c06da50 0x7fcc7c14bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57950/0 (socket says 172.21.15.96:57950) 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 -- 172.21.15.96:0/3967895809 learned_addr learned my addr 172.21.15.96:0/3967895809 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc81ee9640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc7c15e750 0x7fcc7c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc80ee7640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0b3180 0x7fcc7c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 -- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc7c15e750 msgr2=0x7fcc7c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc7c15e750 0x7fcc7c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 -- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0b3180 msgr2=0x7fcc7c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0b3180 0x7fcc7c15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 -- 172.21.15.96:0/3967895809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c155430 con 0x7fcc7c06da50 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc80ee7640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0b3180 0x7fcc7c15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc816e8640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c06da50 0x7fcc7c14bb50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcc780058d0 tx=0x7fcc7800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78017070 con 0x7fcc7c06da50 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc78009d20 con 0x7fcc7c06da50 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c155150 con 0x7fcc7c06da50 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780106c0 con 0x7fcc7c06da50 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c152dd0 con 0x7fcc7c06da50 2024-09-21T02:12:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc7c0b3590 con 0x7fcc7c06da50 2024-09-21T02:12:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcc7801a070 con 0x7fcc7c06da50 2024-09-21T02:12:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fcc60073200 0x7fcc60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fcc78094670 con 0x7fcc7c06da50 2024-09-21T02:12:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7fcc600796f0 0x7fcc6007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 --> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc6007c250 con 0x7fcc600796f0 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcc78094b00 con 0x7fcc7c06da50 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.735+0000 7fcc80ee7640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fcc60073200 0x7fcc60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.739+0000 7fcc81ee9640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7fcc600796f0 0x7fcc6007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.739+0000 7fcc81ee9640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7fcc600796f0 0x7fcc6007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.739+0000 7fcc80ee7640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fcc60073200 0x7fcc60075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0af000 tx=0x7fcc6c019660 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.739+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== osd.3 v2:172.21.15.96:6808/3409109307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc6007c250 con 0x7fcc600796f0 2024-09-21T02:12:40.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 --> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcc7c157dc0 con 0x7fcc600796f0 2024-09-21T02:12:40.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc71ffb640 1 -- 172.21.15.96:0/3967895809 <== osd.3 v2:172.21.15.96:6808/3409109307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcc7c157dc0 con 0x7fcc600796f0 2024-09-21T02:12:40.756 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:40.756 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7fcc600796f0 msgr2=0x7fcc6007bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:6808/3409109307,v1:172.21.15.96:6809/3409109307] conn(0x7fcc600796f0 0x7fcc6007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fcc60073200 msgr2=0x7fcc60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fcc60073200 0x7fcc60075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0af000 tx=0x7fcc6c019660 comp rx=0 tx=0).stop 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c06da50 msgr2=0x7fcc7c14bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 --2- 172.21.15.96:0/3967895809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7c06da50 0x7fcc7c14bb50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcc780058d0 tx=0x7fcc7800dde0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc81ee9640 1 -- 172.21.15.96:0/3967895809 reap_dead start 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 shutdown_connections 2024-09-21T02:12:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 >> 172.21.15.96:0/3967895809 conn(0x7fcc7c0d42b0 msgr2=0x7fcc7c160440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 shutdown_connections 2024-09-21T02:12:40.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.751+0000 7fcc8394c640 1 -- 172.21.15.96:0/3967895809 wait complete. 2024-09-21T02:12:40.773 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T02:12:40.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 -- 172.21.15.96:0/3273152013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15d5a0 msgr2=0x7efebc074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 --2- 172.21.15.96:0/3273152013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15d5a0 0x7efebc074450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efebc0754c0 tx=0x7efeac01c600 comp rx=0 tx=0).stop 2024-09-21T02:12:40.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 -- 172.21.15.96:0/3273152013 shutdown_connections 2024-09-21T02:12:40.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 --2- 172.21.15.96:0/3273152013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15d5a0 0x7efebc074450 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 --2- 172.21.15.96:0/3273152013 >> v2:172.21.15.122:3300/0 conn(0x7efebc15b4a0 0x7efebc15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 -- 172.21.15.96:0/3273152013 >> 172.21.15.96:0/3273152013 conn(0x7efebc0b5170 msgr2=0x7efebc0b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:40.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 -- 172.21.15.96:0/3273152013 shutdown_connections 2024-09-21T02:12:40.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 -- 172.21.15.96:0/3273152013 wait complete. 2024-09-21T02:12:40.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 Processor -- start 2024-09-21T02:12:40.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.915+0000 7efec27ac640 1 -- start start 2024-09-21T02:12:40.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 0x7efebc0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efebc15c950 0x7efebc156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc15d5a0 0x7efebc164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efebc168b50 con 0x7efebc15b4a0 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1689d0 con 0x7efebc15d5a0 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efebc168cd0 con 0x7efebc15c950 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 0x7efebc0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 0x7efebc0ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57970/0 (socket says 172.21.15.96:57970) 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 -- 172.21.15.96:0/615257152 learned_addr learned my addr 172.21.15.96:0/615257152 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec0d49640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc15d5a0 0x7efebc164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebb7fe640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efebc15c950 0x7efebc156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 -- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efebc15c950 msgr2=0x7efebc156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efebc15c950 0x7efebc156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 -- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc15d5a0 msgr2=0x7efebc164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc15d5a0 0x7efebc164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 -- 172.21.15.96:0/615257152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc09dea0 con 0x7efebc15b4a0 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec0d49640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc15d5a0 0x7efebc164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:40.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efebbfff640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 0x7efebc0ccf60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d740 tx=0x7efeb400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4009c30 con 0x7efebc15b4a0 2024-09-21T02:12:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeb400de50 con 0x7efebc15b4a0 2024-09-21T02:12:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc09e180 con 0x7efebc15b4a0 2024-09-21T02:12:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4004280 con 0x7efebc15b4a0 2024-09-21T02:12:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc0cc760 con 0x7efebc15b4a0 2024-09-21T02:12:40.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc15c410 con 0x7efebc15b4a0 2024-09-21T02:12:40.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.919+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efeb4003600 con 0x7efebc15b4a0 2024-09-21T02:12:40.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.923+0000 7efeb8ff9640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efea8073200 0x7efea8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:40.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.923+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7efeb40949c0 con 0x7efebc15b4a0 2024-09-21T02:12:40.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.923+0000 7efebb7fe640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efea8073200 0x7efea8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:40.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.923+0000 7efebb7fe640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efea8073200 0x7efea8075710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efebc1660a0 tx=0x7efeb00094d0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:40.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:40.927+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efeb405fcc0 con 0x7efebc15b4a0 2024-09-21T02:12:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.083+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7efebc069900 con 0x7efea8073200 2024-09-21T02:12:41.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efeb8ff9640 1 -- 172.21.15.96:0/615257152 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efebc069900 con 0x7efea8073200 2024-09-21T02:12:41.090 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efea8073200 msgr2=0x7efea8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efea8073200 0x7efea8075710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efebc1660a0 tx=0x7efeb00094d0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 msgr2=0x7efebc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 0x7efebc0ccf60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d740 tx=0x7efeb400dc00 comp rx=0 tx=0).stop 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 shutdown_connections 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7efea8073200 0x7efea8075710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc15d5a0 0x7efebc164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efebc15c950 0x7efebc156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 --2- 172.21.15.96:0/615257152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efebc15b4a0 0x7efebc0ccf60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 >> 172.21.15.96:0/615257152 conn(0x7efebc0b5170 msgr2=0x7efebc0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 shutdown_connections 2024-09-21T02:12:41.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.087+0000 7efec27ac640 1 -- 172.21.15.96:0/615257152 wait complete. 2024-09-21T02:12:41.111 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T02:12:41.111 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:41.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 -- 172.21.15.96:0/1754744956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15ba70 msgr2=0x7f030c15be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 --2- 172.21.15.96:0/1754744956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15ba70 0x7f030c15be80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f030c070f60 tx=0x7f030401ccc0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 -- 172.21.15.96:0/1754744956 shutdown_connections 2024-09-21T02:12:41.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 --2- 172.21.15.96:0/1754744956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15ba70 0x7f030c15be80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 --2- 172.21.15.96:0/1754744956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f030c15f060 0x7f030c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 --2- 172.21.15.96:0/1754744956 >> v2:172.21.15.122:3300/0 conn(0x7f030c1523d0 0x7f030c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 -- 172.21.15.96:0/1754744956 >> 172.21.15.96:0/1754744956 conn(0x7f030c0d05a0 msgr2=0x7f030c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 -- 172.21.15.96:0/1754744956 shutdown_connections 2024-09-21T02:12:41.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.251+0000 7f031449f640 1 -- 172.21.15.96:0/1754744956 wait complete. 2024-09-21T02:12:41.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 Processor -- start 2024-09-21T02:12:41.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 -- start start 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f030c1523d0 0x7f030c142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f030c15ba70 0x7f030c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15f060 0x7f030c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f030c073d60 con 0x7f030c15f060 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f030c073be0 con 0x7f030c15ba70 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f030c073ee0 con 0x7f030c1523d0 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15f060 0x7f030c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15f060 0x7f030c150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57990/0 (socket says 172.21.15.96:57990) 2024-09-21T02:12:41.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 -- 172.21.15.96:0/4236462812 learned_addr learned my addr 172.21.15.96:0/4236462812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0311a3a640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f030c15ba70 0x7f030c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031223b640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f030c1523d0 0x7f030c142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 -- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f030c1523d0 msgr2=0x7f030c142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f030c1523d0 0x7f030c142c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 -- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f030c15ba70 msgr2=0x7f030c1505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f030c15ba70 0x7f030c1505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 -- 172.21.15.96:0/4236462812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c14e3b0 con 0x7f030c15f060 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0311a3a640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f030c15ba70 0x7f030c1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0312a3c640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15f060 0x7f030c150b30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f030401b370 tx=0x7f0304005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040bc070 con 0x7f030c15f060 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03040042f0 con 0x7f030c15f060 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c156760 con 0x7f030c15f060 2024-09-21T02:12:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040b2040 con 0x7f030c15f060 2024-09-21T02:12:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.255+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c156d20 con 0x7f030c15f060 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f030c074400 con 0x7f030c15f060 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f03040bf070 con 0x7f030c15f060 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0302ffd640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f02f0073200 0x7f02f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f030412e140 con 0x7f030c15f060 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0302ffd640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7f02f00796f0 0x7f02f007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 --> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02f007c250 con 0x7f02f00796f0 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f030412e5d0 con 0x7f030c15f060 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f031223b640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f02f0073200 0x7f02f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0311a3a640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7f02f00796f0 0x7f02f007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f031223b640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f02f0073200 0x7f02f0075710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0308009980 tx=0x7f0308005ca0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.259+0000 7f0311a3a640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7f02f00796f0 0x7f02f007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.263+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== osd.4 v2:172.21.15.122:6802/567414413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02f007c250 con 0x7f02f00796f0 2024-09-21T02:12:41.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 --> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f030c165fc0 con 0x7f02f00796f0 2024-09-21T02:12:41.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f0302ffd640 1 -- 172.21.15.96:0/4236462812 <== osd.4 v2:172.21.15.122:6802/567414413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f030c165fc0 con 0x7f02f00796f0 2024-09-21T02:12:41.280 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:41.280 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:41.281 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:41.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7f02f00796f0 msgr2=0x7f02f007bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6802/567414413,v1:172.21.15.122:6803/567414413] conn(0x7f02f00796f0 0x7f02f007bb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f02f0073200 msgr2=0x7f02f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f02f0073200 0x7f02f0075710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0308009980 tx=0x7f0308005ca0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15f060 msgr2=0x7f030c150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 --2- 172.21.15.96:0/4236462812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c15f060 0x7f030c150b30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f030401b370 tx=0x7f0304005b10 comp rx=0 tx=0).stop 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f0312a3c640 1 -- 172.21.15.96:0/4236462812 reap_dead start 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 shutdown_connections 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 >> 172.21.15.96:0/4236462812 conn(0x7f030c0d05a0 msgr2=0x7f030c0700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 shutdown_connections 2024-09-21T02:12:41.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.275+0000 7f031449f640 1 -- 172.21.15.96:0/4236462812 wait complete. 2024-09-21T02:12:41.298 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 -- 172.21.15.96:0/2372765398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 msgr2=0x7fe8d406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/2372765398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d406dea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40cca40 tx=0x7fe8c401cd80 comp rx=0 tx=0).stop 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 -- 172.21.15.96:0/2372765398 shutdown_connections 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/2372765398 >> v2:172.21.15.122:3300/0 conn(0x7fe8d415c3e0 0x7fe8d40cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/2372765398 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8d406e3e0 0x7fe8d415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/2372765398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d406dea0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 -- 172.21.15.96:0/2372765398 >> 172.21.15.96:0/2372765398 conn(0x7fe8d406eea0 msgr2=0x7fe8d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 -- 172.21.15.96:0/2372765398 shutdown_connections 2024-09-21T02:12:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 -- 172.21.15.96:0/2372765398 wait complete. 2024-09-21T02:12:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 Processor -- start 2024-09-21T02:12:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.443+0000 7fe8d9e82640 1 -- start start 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8d406e3e0 0x7fe8d4156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d414d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8d415c3e0 0x7fe8d4150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40caa00 con 0x7fe8d40ced00 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40ca880 con 0x7fe8d406e3e0 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40cab80 con 0x7fe8d415c3e0 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d414d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d414d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58006/0 (socket says 172.21.15.96:58006) 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 -- 172.21.15.96:0/1535314269 learned_addr learned my addr 172.21.15.96:0/1535314269 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d37fe640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8d406e3e0 0x7fe8d4156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d3fff640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8d415c3e0 0x7fe8d4150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 -- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8d415c3e0 msgr2=0x7fe8d4150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8d415c3e0 0x7fe8d4150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 -- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8d406e3e0 msgr2=0x7fe8d4156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8d406e3e0 0x7fe8d4156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 -- 172.21.15.96:0/1535314269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d4156670 con 0x7fe8d40ced00 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d2ffd640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d414d780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d660 tx=0x7fe8c800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8017070 con 0x7fe8d40ced00 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8c8005310 con 0x7fe8d40ced00 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8009820 con 0x7fe8d40ced00 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41f90f0 con 0x7fe8d40ced00 2024-09-21T02:12:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d4151350 con 0x7fe8d40ced00 2024-09-21T02:12:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d40cb150 con 0x7fe8d40ced00 2024-09-21T02:12:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe8c80099c0 con 0x7fe8d40ced00 2024-09-21T02:12:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8bbfff640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe8c0073200 0x7fe8c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.447+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fe8c8094440 con 0x7fe8d40ced00 2024-09-21T02:12:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.451+0000 7fe8d37fe640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe8c0073200 0x7fe8c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.451+0000 7fe8d37fe640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe8c0073200 0x7fe8c0075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40af000 tx=0x7fe8c401b5c0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.451+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe8c805f7d0 con 0x7fe8d40ced00 2024-09-21T02:12:41.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.611+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe8d414e2e0 con 0x7fe8c0073200 2024-09-21T02:12:41.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.611+0000 7fe8bbfff640 1 -- 172.21.15.96:0/1535314269 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe8d414e2e0 con 0x7fe8c0073200 2024-09-21T02:12:41.618 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe8c0073200 msgr2=0x7fe8c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe8c0073200 0x7fe8c0075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40af000 tx=0x7fe8c401b5c0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 msgr2=0x7fe8d414d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d414d780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d660 tx=0x7fe8c800db20 comp rx=0 tx=0).stop 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 shutdown_connections 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fe8c0073200 0x7fe8c0075710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8d415c3e0 0x7fe8d4150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d40ced00 0x7fe8d414d780 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 --2- 172.21.15.96:0/1535314269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8d406e3e0 0x7fe8d4156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 >> 172.21.15.96:0/1535314269 conn(0x7fe8d406eea0 msgr2=0x7fe8d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 shutdown_connections 2024-09-21T02:12:41.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.615+0000 7fe8d9e82640 1 -- 172.21.15.96:0/1535314269 wait complete. 2024-09-21T02:12:41.638 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T02:12:41.638 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:41.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 -- 172.21.15.96:0/76628197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 msgr2=0x7f57e815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 --2- 172.21.15.96:0/76628197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e815d370 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009c80 tx=0x7f57e001cd10 comp rx=0 tx=0).stop 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 -- 172.21.15.96:0/76628197 shutdown_connections 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 --2- 172.21.15.96:0/76628197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e815d370 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 --2- 172.21.15.96:0/76628197 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57e8074420 0x7f57e815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 -- 172.21.15.96:0/76628197 >> 172.21.15.96:0/76628197 conn(0x7f57e806fdb0 msgr2=0x7f57e80701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 -- 172.21.15.96:0/76628197 shutdown_connections 2024-09-21T02:12:41.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 -- 172.21.15.96:0/76628197 wait complete. 2024-09-21T02:12:41.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 Processor -- start 2024-09-21T02:12:41.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.779+0000 7f57eff9e640 1 -- start start 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57e815b2d0 0x7f57e80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e814d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57e80d0570 0x7f57e814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8169170 con 0x7f57e815cf60 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8168ff0 con 0x7f57e80d0570 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f57e81692f0 con 0x7f57e815b2d0 2024-09-21T02:12:41.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e814d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e814d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58016/0 (socket says 172.21.15.96:58016) 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 -- 172.21.15.96:0/2490499692 learned_addr learned my addr 172.21.15.96:0/2490499692 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ee53b640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57e80d0570 0x7f57e814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57edd3a640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57e815b2d0 0x7f57e80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 -- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57e815b2d0 msgr2=0x7f57e80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57e815b2d0 0x7f57e80cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 -- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57e80d0570 msgr2=0x7f57e814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57e80d0570 0x7f57e814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 -- 172.21.15.96:0/2490499692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e81fc410 con 0x7f57e815cf60 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57edd3a640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57e815b2d0 0x7f57e80cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ed539640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e814d740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f57d80058d0 tx=0x7f57d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57e815cf60 2024-09-21T02:12:41.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57d8009d20 con 0x7f57e815cf60 2024-09-21T02:12:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e8152ba0 con 0x7f57e815cf60 2024-09-21T02:12:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80106c0 con 0x7f57e815cf60 2024-09-21T02:12:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e8153160 con 0x7f57e815cf60 2024-09-21T02:12:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57e8153540 con 0x7f57e815cf60 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57d801a070 con 0x7f57e815cf60 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f57cc073230 0x7f57cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f57d8094280 con 0x7f57e815cf60 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f57cc079720 0x7f57cc07bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 --> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57d801ae50 con 0x7f57cc079720 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f57d809b050 con 0x7f57e815cf60 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57edd3a640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f57cc073230 0x7f57cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.783+0000 7f57ee53b640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f57cc079720 0x7f57cc07bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.787+0000 7f57edd3a640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f57cc073230 0x7f57cc075740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000f70 tx=0x7f57e4009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.787+0000 7f57ee53b640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f57cc079720 0x7f57cc07bbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.787+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== osd.5 v2:172.21.15.122:6810/4177593286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57d801ae50 con 0x7f57cc079720 2024-09-21T02:12:41.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.799+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 --> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57e80cfa20 con 0x7f57cc079720 2024-09-21T02:12:41.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.799+0000 7f57de7fc640 1 -- 172.21.15.96:0/2490499692 <== osd.5 v2:172.21.15.122:6810/4177593286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f57e80cfa20 con 0x7f57cc079720 2024-09-21T02:12:41.806 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:41.806 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:41.806 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.799+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f57cc079720 msgr2=0x7f57cc07bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.799+0000 7f57eff9e640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6810/4177593286,v1:172.21.15.122:6811/4177593286] conn(0x7f57cc079720 0x7f57cc07bbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f57cc073230 msgr2=0x7f57cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f57cc073230 0x7f57cc075740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000f70 tx=0x7f57e4009040 comp rx=0 tx=0).stop 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 msgr2=0x7f57e814d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 --2- 172.21.15.96:0/2490499692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57e815cf60 0x7f57e814d740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f57d80058d0 tx=0x7f57d800dde0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57ee53b640 1 -- 172.21.15.96:0/2490499692 reap_dead start 2024-09-21T02:12:41.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 shutdown_connections 2024-09-21T02:12:41.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 >> 172.21.15.96:0/2490499692 conn(0x7f57e806fdb0 msgr2=0x7f57e81555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 shutdown_connections 2024-09-21T02:12:41.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.803+0000 7f57eff9e640 1 -- 172.21.15.96:0/2490499692 wait complete. 2024-09-21T02:12:41.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T02:12:41.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- 172.21.15.96:0/2370071988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 msgr2=0x7f6050003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 --2- 172.21.15.96:0/2370071988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f6050003eb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f605c009ee0 tx=0x7f605c01d610 comp rx=0 tx=0).stop 2024-09-21T02:12:41.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6062d9d640 1 -- 172.21.15.96:0/2370071988 reap_dead start 2024-09-21T02:12:41.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- 172.21.15.96:0/2370071988 shutdown_connections 2024-09-21T02:12:41.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- 172.21.15.96:0/2370071988 >> 172.21.15.96:0/2370071988 conn(0x7f60640d2100 msgr2=0x7f60640d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- 172.21.15.96:0/2370071988 shutdown_connections 2024-09-21T02:12:41.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- 172.21.15.96:0/2370071988 wait complete. 2024-09-21T02:12:41.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 Processor -- start 2024-09-21T02:12:41.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- start start 2024-09-21T02:12:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f60641551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f606406dfa0 0x7f60641ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f606415db50 0x7f60641ec640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6064071500 con 0x7f6050003a40 2024-09-21T02:12:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6064071380 con 0x7f606415db50 2024-09-21T02:12:41.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f6068c3a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6064071680 con 0x7f606406dfa0 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f606259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f60641551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f606259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f60641551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58042/0 (socket says 172.21.15.96:58042) 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.971+0000 7f606259c640 1 -- 172.21.15.96:0/1158972050 learned_addr learned my addr 172.21.15.96:0/1158972050 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6061d9b640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f606406dfa0 0x7f60641ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6062d9d640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f606415db50 0x7f60641ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f606259c640 1 -- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f606406dfa0 msgr2=0x7f60641ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f606259c640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f606406dfa0 0x7f60641ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f606259c640 1 -- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f606415db50 msgr2=0x7f60641ec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f606259c640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f606415db50 0x7f60641ec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f606259c640 1 -- 172.21.15.96:0/1158972050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606406e710 con 0x7f6050003a40 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6061d9b640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f606406dfa0 0x7f60641ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f606259c640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f60641551e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f605c0ae040 tx=0x7f605c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c005300 con 0x7f6050003a40 2024-09-21T02:12:41.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f605c005590 con 0x7f6050003a40 2024-09-21T02:12:41.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606406e9f0 con 0x7f6050003a40 2024-09-21T02:12:41.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c005980 con 0x7f6050003a40 2024-09-21T02:12:41.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606406efb0 con 0x7f6050003a40 2024-09-21T02:12:41.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6064071be0 con 0x7f6050003a40 2024-09-21T02:12:41.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f605c0ac450 con 0x7f6050003a40 2024-09-21T02:12:41.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f605affd640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6044073200 0x7f6044075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f605c130000 con 0x7f6050003a40 2024-09-21T02:12:41.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.975+0000 7f6061d9b640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6044073200 0x7f6044075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.979+0000 7f6061d9b640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6044073200 0x7f6044075710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f604c001870 tx=0x7f604c001e40 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:41.979+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f605c0c6060 con 0x7f6050003a40 2024-09-21T02:12:42.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.139+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f6064155720 con 0x7f6044073200 2024-09-21T02:12:42.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.139+0000 7f605affd640 1 -- 172.21.15.96:0/1158972050 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6064155720 con 0x7f6044073200 2024-09-21T02:12:42.144 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T02:12:42.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6044073200 msgr2=0x7f6044075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6044073200 0x7f6044075710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f604c001870 tx=0x7f604c001e40 comp rx=0 tx=0).stop 2024-09-21T02:12:42.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 msgr2=0x7f60641551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f60641551e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f605c0ae040 tx=0x7f605c009b30 comp rx=0 tx=0).stop 2024-09-21T02:12:42.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 shutdown_connections 2024-09-21T02:12:42.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6044073200 0x7f6044075710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f606415db50 0x7f60641ec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f606406dfa0 0x7f60641ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 --2- 172.21.15.96:0/1158972050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6050003a40 0x7f60641551e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 >> 172.21.15.96:0/1158972050 conn(0x7f60640d2100 msgr2=0x7f60640b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 shutdown_connections 2024-09-21T02:12:42.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.143+0000 7f6068c3a640 1 -- 172.21.15.96:0/1158972050 wait complete. 2024-09-21T02:12:42.165 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T02:12:42.165 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 -- 172.21.15.96:0/3080067625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc0ccf30 msgr2=0x7f44bc0cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 --2- 172.21.15.96:0/3080067625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc0ccf30 0x7f44bc0cf140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009c80 tx=0x7f44b401cc10 comp rx=0 tx=0).stop 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 -- 172.21.15.96:0/3080067625 shutdown_connections 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 --2- 172.21.15.96:0/3080067625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc0ccf30 0x7f44bc0cf140 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 --2- 172.21.15.96:0/3080067625 >> v2:172.21.15.122:3300/0 conn(0x7f44bc0ccb20 0x7f44bc0cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 -- 172.21.15.96:0/3080067625 >> 172.21.15.96:0/3080067625 conn(0x7f44bc0b3e50 msgr2=0x7f44bc0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 -- 172.21.15.96:0/3080067625 shutdown_connections 2024-09-21T02:12:42.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 -- 172.21.15.96:0/3080067625 wait complete. 2024-09-21T02:12:42.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 Processor -- start 2024-09-21T02:12:42.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.323+0000 7f44c394c640 1 -- start start 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44bc0ccb20 0x7f44bc06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f44bc0ccf30 0x7f44bc15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc160010 0x7f44bc15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1534d0 con 0x7f44bc160010 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc153350 con 0x7f44bc0ccb20 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f44bc153650 con 0x7f44bc0ccf30 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc160010 0x7f44bc15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc160010 0x7f44bc15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58064/0 (socket says 172.21.15.96:58064) 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 -- 172.21.15.96:0/1892745969 learned_addr learned my addr 172.21.15.96:0/1892745969 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c0ee7640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f44bc0ccf30 0x7f44bc15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c16e8640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44bc0ccb20 0x7f44bc06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 -- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f44bc0ccf30 msgr2=0x7f44bc15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f44bc0ccf30 0x7f44bc15b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 -- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44bc0ccb20 msgr2=0x7f44bc06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44bc0ccb20 0x7f44bc06fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 -- 172.21.15.96:0/1892745969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc06ffb0 con 0x7f44bc160010 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c16e8640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44bc0ccb20 0x7f44bc06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c1ee9640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc160010 0x7f44bc15ba00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f44b401d4b0 tx=0x7f44b40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40bb070 con 0x7f44bc160010 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44b4004d40 con 0x7f44bc160010 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40b2040 con 0x7f44bc160010 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc14f390 con 0x7f44bc160010 2024-09-21T02:12:42.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc153f50 con 0x7f44bc160010 2024-09-21T02:12:42.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44bc15d2b0 con 0x7f44bc160010 2024-09-21T02:12:42.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f44b40be070 con 0x7f44bc160010 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f44a0073230 0x7f44a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f44b412e2c0 con 0x7f44bc160010 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7f44a0079720 0x7f44a007bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 --> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44a007c280 con 0x7f44a0079720 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.327+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f44b412e750 con 0x7f44bc160010 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.331+0000 7f44c16e8640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f44a0073230 0x7f44a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.331+0000 7f44c0ee7640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7f44a0079720 0x7f44a007bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.331+0000 7f44c0ee7640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7f44a0079720 0x7f44a007bbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.331+0000 7f44c16e8640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f44a0073230 0x7f44a0075740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44b800a850 tx=0x7f44b8008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.331+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== osd.6 v2:172.21.15.122:6821/3298887226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a007c280 con 0x7f44a0079720 2024-09-21T02:12:42.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 --> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f44bc06e290 con 0x7f44a0079720 2024-09-21T02:12:42.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44b1ffb640 1 -- 172.21.15.96:0/1892745969 <== osd.6 v2:172.21.15.122:6821/3298887226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f44bc06e290 con 0x7f44a0079720 2024-09-21T02:12:42.351 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:42.352 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:42.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7f44a0079720 msgr2=0x7f44a007bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6821/3298887226,v1:172.21.15.122:6823/3298887226] conn(0x7f44a0079720 0x7f44a007bbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f44a0073230 msgr2=0x7f44a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f44a0073230 0x7f44a0075740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44b800a850 tx=0x7f44b8008040 comp rx=0 tx=0).stop 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc160010 msgr2=0x7f44bc15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 --2- 172.21.15.96:0/1892745969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44bc160010 0x7f44bc15ba00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f44b401d4b0 tx=0x7f44b40093d0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c1ee9640 1 -- 172.21.15.96:0/1892745969 reap_dead start 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 shutdown_connections 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 >> 172.21.15.96:0/1892745969 conn(0x7f44bc0b3e50 msgr2=0x7f44bc0b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 shutdown_connections 2024-09-21T02:12:42.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.347+0000 7f44c394c640 1 -- 172.21.15.96:0/1892745969 wait complete. 2024-09-21T02:12:42.371 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T02:12:42.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 -- 172.21.15.96:0/2669841795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb60415c5b0 msgr2=0x7fb6040cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 --2- 172.21.15.96:0/2669841795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb60415c5b0 0x7fb6040cbaf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb60406a1c0 tx=0x7fb5fc01cd80 comp rx=0 tx=0).stop 2024-09-21T02:12:42.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 -- 172.21.15.96:0/2669841795 shutdown_connections 2024-09-21T02:12:42.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 --2- 172.21.15.96:0/2669841795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb60415c5b0 0x7fb6040cbaf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 --2- 172.21.15.96:0/2669841795 >> v2:172.21.15.122:3300/0 conn(0x7fb6040ced00 0x7fb60406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 -- 172.21.15.96:0/2669841795 >> 172.21.15.96:0/2669841795 conn(0x7fb60406eea0 msgr2=0x7fb60406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 -- 172.21.15.96:0/2669841795 shutdown_connections 2024-09-21T02:12:42.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.515+0000 7fb60a800640 1 -- 172.21.15.96:0/2669841795 wait complete. 2024-09-21T02:12:42.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 Processor -- start 2024-09-21T02:12:42.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 -- start start 2024-09-21T02:12:42.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60406e3e0 0x7fb604149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 0x7fb604165190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb60415c5b0 0x7fb6041656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040ce120 con 0x7fb6040ced00 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040cdfa0 con 0x7fb60406e3e0 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb6040ce2a0 con 0x7fb60415c5b0 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 0x7fb604165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 0x7fb604165190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58094/0 (socket says 172.21.15.96:58094) 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 -- 172.21.15.96:0/401813347 learned_addr learned my addr 172.21.15.96:0/401813347 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb603fff640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60406e3e0 0x7fb604149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb608d9d640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb60415c5b0 0x7fb6041656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 -- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb60415c5b0 msgr2=0x7fb6041656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb60415c5b0 0x7fb6041656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 -- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60406e3e0 msgr2=0x7fb604149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60406e3e0 0x7fb604149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 -- 172.21.15.96:0/401813347 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60414ec10 con 0x7fb6040ced00 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb6037fe640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 0x7fb604165190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c3f0 tx=0x7fb5f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8017070 con 0x7fb6040ced00 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6041517e0 con 0x7fb6040ced00 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f8003600 con 0x7fb6040ced00 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8003a50 con 0x7fb6040ced00 2024-09-21T02:12:42.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.519+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60414eed0 con 0x7fb6040ced00 2024-09-21T02:12:42.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.523+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6040cbaf0 con 0x7fb6040ced00 2024-09-21T02:12:42.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.523+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5f801a070 con 0x7fb6040ced00 2024-09-21T02:12:42.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.523+0000 7fb600ff9640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb5f0073200 0x7fb5f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.523+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fb5f80937a0 con 0x7fb6040ced00 2024-09-21T02:12:42.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.523+0000 7fb603fff640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb5f0073200 0x7fb5f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.523+0000 7fb603fff640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb5f0073200 0x7fb5f0075710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40058d0 tx=0x7fb5f4005860 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.527+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb5f8021020 con 0x7fb6040ced00 2024-09-21T02:12:42.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.687+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb6041f9290 con 0x7fb5f0073200 2024-09-21T02:12:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.687+0000 7fb600ff9640 1 -- 172.21.15.96:0/401813347 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb6041f9290 con 0x7fb5f0073200 2024-09-21T02:12:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb5f0073200 msgr2=0x7fb5f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb5f0073200 0x7fb5f0075710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40058d0 tx=0x7fb5f4005860 comp rx=0 tx=0).stop 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 msgr2=0x7fb604165190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 0x7fb604165190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c3f0 tx=0x7fb5f800c8b0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 shutdown_connections 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7fb5f0073200 0x7fb5f0075710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb60415c5b0 0x7fb6041656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6040ced00 0x7fb604165190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 --2- 172.21.15.96:0/401813347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60406e3e0 0x7fb604149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 >> 172.21.15.96:0/401813347 conn(0x7fb60406eea0 msgr2=0x7fb60415ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 shutdown_connections 2024-09-21T02:12:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.691+0000 7fb60a800640 1 -- 172.21.15.96:0/401813347 wait complete. 2024-09-21T02:12:42.714 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T02:12:42.714 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:12:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 -- 172.21.15.96:0/2426946394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc0b3590 msgr2=0x7f1edc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 --2- 172.21.15.96:0/2426946394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc0b3590 0x7f1edc06e160 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009c80 tx=0x7f1ed401d7d0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 -- 172.21.15.96:0/2426946394 shutdown_connections 2024-09-21T02:12:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 --2- 172.21.15.96:0/2426946394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc0b3590 0x7f1edc06e160 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 --2- 172.21.15.96:0/2426946394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1edc15e750 0x7f1edc0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 -- 172.21.15.96:0/2426946394 >> 172.21.15.96:0/2426946394 conn(0x7f1edc0d42b0 msgr2=0x7f1edc0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 -- 172.21.15.96:0/2426946394 shutdown_connections 2024-09-21T02:12:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 -- 172.21.15.96:0/2426946394 wait complete. 2024-09-21T02:12:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 Processor -- start 2024-09-21T02:12:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.871+0000 7f1ee3588640 1 -- start start 2024-09-21T02:12:42.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1edc0b3180 0x7f1edc155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1edc0b3590 0x7f1edc1fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc15e750 0x7f1edc152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc0b8d10 con 0x7f1edc15e750 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc0b8b90 con 0x7f1edc0b3590 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc0b8e90 con 0x7f1edc0b3180 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc15e750 0x7f1edc152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc15e750 0x7f1edc152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58112/0 (socket says 172.21.15.96:58112) 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 -- 172.21.15.96:0/3027886547 learned_addr learned my addr 172.21.15.96:0/3027886547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee0b23640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1edc0b3590 0x7f1edc1fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1324640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1edc0b3180 0x7f1edc155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 -- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1edc0b3180 msgr2=0x7f1edc155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1edc0b3180 0x7f1edc155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 -- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1edc0b3590 msgr2=0x7f1edc1fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1edc0b3590 0x7f1edc1fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 -- 172.21.15.96:0/3027886547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc156aa0 con 0x7f1edc15e750 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee0b23640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1edc0b3590 0x7f1edc1fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1b25640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc15e750 0x7f1edc152200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009bd0 tx=0x7f1ed40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40b2040 con 0x7f1edc15e750 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc155a00 con 0x7f1edc15e750 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed4004d40 con 0x7f1edc15e750 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40042a0 con 0x7f1edc15e750 2024-09-21T02:12:42.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc155fc0 con 0x7f1edc15e750 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1edc0b9340 con 0x7f1edc15e750 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1ed40bd060 con 0x7f1edc15e750 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1ec0073200 0x7f1ec0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f1ed412e610 con 0x7f1edc15e750 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f1ec00796f0 0x7f1ec007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 --> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ed40b3e50 con 0x7f1ec00796f0 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1ed412eaa0 con 0x7f1edc15e750 2024-09-21T02:12:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee1324640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1ec0073200 0x7f1ec0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.875+0000 7f1ee0b23640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f1ec00796f0 0x7f1ec007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.879+0000 7f1ee0b23640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f1ec00796f0 0x7f1ec007bb80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.879+0000 7f1ee1324640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1ec0073200 0x7f1ec0075710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80058d0 tx=0x7f1ed8005bd0 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.879+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== osd.7 v2:172.21.15.122:6818/1732471888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ed40b3e50 con 0x7f1ec00796f0 2024-09-21T02:12:42.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.891+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 --> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1edc155790 con 0x7f1ec00796f0 2024-09-21T02:12:42.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ed1ffb640 1 -- 172.21.15.96:0/3027886547 <== osd.7 v2:172.21.15.122:6818/1732471888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1edc155790 con 0x7f1ec00796f0 2024-09-21T02:12:42.899 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-21T02:12:42.899 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:12:42.899 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f1ec00796f0 msgr2=0x7f1ec007bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6818/1732471888,v1:172.21.15.122:6819/1732471888] conn(0x7f1ec00796f0 0x7f1ec007bb80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1ec0073200 msgr2=0x7f1ec0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f1ec0073200 0x7f1ec0075710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80058d0 tx=0x7f1ed8005bd0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc15e750 msgr2=0x7f1edc152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 --2- 172.21.15.96:0/3027886547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1edc15e750 0x7f1edc152200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009bd0 tx=0x7f1ed40092c0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee1b25640 1 -- 172.21.15.96:0/3027886547 reap_dead start 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 shutdown_connections 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 >> 172.21.15.96:0/3027886547 conn(0x7f1edc0d42b0 msgr2=0x7f1edc1574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 shutdown_connections 2024-09-21T02:12:42.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:42.895+0000 7f1ee3588640 1 -- 172.21.15.96:0/3027886547 wait complete. 2024-09-21T02:12:42.918 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/1260407019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 msgr2=0x7f6e780d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/1260407019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e780d2460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009900 tx=0x7f6e6c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/1260407019 shutdown_connections 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/1260407019 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780d29a0 0x7f6e780d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/1260407019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e780d2460 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/1260407019 >> 172.21.15.96:0/1260407019 conn(0x7f6e780d1b10 msgr2=0x7f6e780d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/1260407019 shutdown_connections 2024-09-21T02:12:43.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/1260407019 wait complete. 2024-09-21T02:12:43.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.063+0000 7f6e7e2ff640 1 Processor -- start 2024-09-21T02:12:43.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- start start 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780cf710 0x7f6e780cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e780d29a0 0x7f6e780bee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e781557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7816c140 con 0x7f6e7815f9a0 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7816bfc0 con 0x7f6e780d29a0 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7816c2c0 con 0x7f6e780cf710 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e781557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e781557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58124/0 (socket says 172.21.15.96:58124) 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 -- 172.21.15.96:0/380460493 learned_addr learned my addr 172.21.15.96:0/380460493 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e777fe640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e780d29a0 0x7f6e780bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e77fff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780cf710 0x7f6e780cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 -- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780cf710 msgr2=0x7f6e780cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780cf710 0x7f6e780cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 -- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e780d29a0 msgr2=0x7f6e780bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e780d29a0 0x7f6e780bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 -- 172.21.15.96:0/380460493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e780ce450 con 0x7f6e7815f9a0 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e77fff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780cf710 0x7f6e780cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7c89c640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e781557b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000cab0 tx=0x7f6e7000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7000ae20 con 0x7f6e7815f9a0 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e7000f560 con 0x7f6e7815f9a0 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7000f9b0 con 0x7f6e7815f9a0 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e780ce170 con 0x7f6e7815f9a0 2024-09-21T02:12:43.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e78156fa0 con 0x7f6e7815f9a0 2024-09-21T02:12:43.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.067+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e780d2db0 con 0x7f6e7815f9a0 2024-09-21T02:12:43.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.071+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6e700213f0 con 0x7f6e7815f9a0 2024-09-21T02:12:43.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.071+0000 7f6e74ff9640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6e64073200 0x7f6e64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.071+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f6e70094d10 con 0x7f6e7815f9a0 2024-09-21T02:12:43.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.071+0000 7f6e77fff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6e64073200 0x7f6e64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.071+0000 7f6e77fff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6e64073200 0x7f6e64075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6e780b7610 tx=0x7f6e68008040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:43.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.075+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6e70014030 con 0x7f6e7815f9a0 2024-09-21T02:12:43.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.235+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f6e780b7db0 con 0x7f6e64073200 2024-09-21T02:12:43.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.235+0000 7f6e74ff9640 1 -- 172.21.15.96:0/380460493 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6e780b7db0 con 0x7f6e64073200 2024-09-21T02:12:43.242 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T02:12:43.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6e64073200 msgr2=0x7f6e64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6e64073200 0x7f6e64075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6e780b7610 tx=0x7f6e68008040 comp rx=0 tx=0).stop 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 msgr2=0x7f6e781557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e781557b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000cab0 tx=0x7f6e7000cf70 comp rx=0 tx=0).stop 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 shutdown_connections 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f6e64073200 0x7f6e64075710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e7815f9a0 0x7f6e781557b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e780d29a0 0x7f6e780bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 --2- 172.21.15.96:0/380460493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e780cf710 0x7f6e780cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 >> 172.21.15.96:0/380460493 conn(0x7f6e780d1b10 msgr2=0x7f6e7815f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 shutdown_connections 2024-09-21T02:12:43.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.239+0000 7f6e7e2ff640 1 -- 172.21.15.96:0/380460493 wait complete. 2024-09-21T02:12:43.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240e4f3640 1 -- 172.21.15.96:0/322853524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f0003a10 msgr2=0x7f23f0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240e4f3640 1 --2- 172.21.15.96:0/322853524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f0003a10 0x7f23f0003e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f23f800bb10 tx=0x7f23f801cd80 comp rx=0 tx=0).stop 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240ca90640 1 -- 172.21.15.96:0/322853524 reap_dead start 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240e4f3640 1 -- 172.21.15.96:0/322853524 shutdown_connections 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240e4f3640 1 -- 172.21.15.96:0/322853524 >> 172.21.15.96:0/322853524 conn(0x7f240806eea0 msgr2=0x7f240806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240e4f3640 1 -- 172.21.15.96:0/322853524 shutdown_connections 2024-09-21T02:12:43.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.395+0000 7f240e4f3640 1 -- 172.21.15.96:0/322853524 wait complete. 2024-09-21T02:12:43.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 Processor -- start 2024-09-21T02:12:43.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- start start 2024-09-21T02:12:43.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23f0003a10 0x7f24081f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 0x7f24081505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2408153b20 0x7f2408153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2408166050 con 0x7f24080b4300 2024-09-21T02:12:43.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2408165ed0 con 0x7f2408153b20 2024-09-21T02:12:43.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f24081661d0 con 0x7f23f0003a10 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 0x7f24081505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 0x7f24081505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58140/0 (socket says 172.21.15.96:58140) 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 -- 172.21.15.96:0/77277533 learned_addr learned my addr 172.21.15.96:0/77277533 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f2407fff640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23f0003a10 0x7f24081f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240ca90640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2408153b20 0x7f2408153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 -- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23f0003a10 msgr2=0x7f24081f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23f0003a10 0x7f24081f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 -- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2408153b20 msgr2=0x7f2408153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2408153b20 0x7f2408153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 -- 172.21.15.96:0/77277533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2408154470 con 0x7f24080b4300 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240ca90640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2408153b20 0x7f2408153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f24077fe640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 0x7f24081505b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00d7b0 tx=0x7f23fc00dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:43.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc018070 con 0x7f24080b4300 2024-09-21T02:12:43.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23fc0052e0 con 0x7f24080b4300 2024-09-21T02:12:43.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24080b3260 con 0x7f24080b4300 2024-09-21T02:12:43.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00de60 con 0x7f24080b4300 2024-09-21T02:12:43.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24080b3820 con 0x7f24080b4300 2024-09-21T02:12:43.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.399+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24080cab60 con 0x7f24080b4300 2024-09-21T02:12:43.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.403+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f23fc01b070 con 0x7f24080b4300 2024-09-21T02:12:43.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.403+0000 7f2404ff9640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f23f4073230 0x7f23f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:43.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.403+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f23fc094030 con 0x7f24080b4300 2024-09-21T02:12:43.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.403+0000 7f2407fff640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f23f4073230 0x7f23f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:43.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.403+0000 7f2407fff640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f23f4073230 0x7f23f4075740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23f80b0000 tx=0x7f23f80a7040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:43.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.407+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2408154470 con 0x7f24080b4300 2024-09-21T02:12:43.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.563+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24080b2fb0 con 0x7f23f4073230 2024-09-21T02:12:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.563+0000 7f2404ff9640 1 -- 172.21.15.96:0/77277533 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13451 (secure 0 0 0) 0x7f24080b2fb0 con 0x7f23f4073230 2024-09-21T02:12:43.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:12:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:12:43.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.563+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f23f4073230 msgr2=0x7f23f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.563+0000 7f240e4f3640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f23f4073230 0x7f23f4075740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23f80b0000 tx=0x7f23f80a7040 comp rx=0 tx=0).stop 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.563+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 msgr2=0x7f24081505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.563+0000 7f240e4f3640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 0x7f24081505b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00d7b0 tx=0x7f23fc00dc70 comp rx=0 tx=0).stop 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 shutdown_connections 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f23f4073230 0x7f23f4075740 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2408153b20 0x7f2408153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24080b4300 0x7f24081505b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 --2- 172.21.15.96:0/77277533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23f0003a10 0x7f24081f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 >> 172.21.15.96:0/77277533 conn(0x7f240806eea0 msgr2=0x7f240815c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 shutdown_connections 2024-09-21T02:12:43.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:12:43.567+0000 7f240e4f3640 1 -- 172.21.15.96:0/77277533 wait complete. 2024-09-21T02:12:43.589 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-21T02:12:42.580578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229852,"kb_used_data":5288,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744868,"statfs":{"total":773094113280,"available":772858744832,"internally_reserved":0,"allocated":5414912,"data_stored":3016408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.916091"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":158,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-21T02:12:34.698374+0000","last_change":"2024-09-21T02:09:02.624317+0000","last_active":"2024-09-21T02:12:34.698374+0000","last_peered":"2024-09-21T02:12:34.698374+0000","last_clean":"2024-09-21T02:12:34.698374+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:12:34.698374+0000","last_undegraded":"2024-09-21T02:12:34.698374+0000","last_fullsized":"2024-09-21T02:12:34.698374+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:09:01.111232+0000","last_clean_scrub_stamp":"2024-09-21T02:09:01.111232+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:21.398601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":819827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31708,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1122304,"data_stored":819827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:12:43.591 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T02:09:01.111232+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=9, tm_sec=1, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=12, tm_sec=37, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:12:43.591 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T02:13:03.592 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/300312963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 msgr2=0x7f8a44003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/300312963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a44003eb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009b60 tx=0x7f8a4801d250 comp rx=0 tx=0).stop 2024-09-21T02:13:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5e3fb640 1 -- 172.21.15.96:0/300312963 reap_dead start 2024-09-21T02:13:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/300312963 shutdown_connections 2024-09-21T02:13:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/300312963 >> 172.21.15.96:0/300312963 conn(0x7f8a5806eea0 msgr2=0x7f8a5806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/300312963 shutdown_connections 2024-09-21T02:13:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/300312963 wait complete. 2024-09-21T02:13:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 Processor -- start 2024-09-21T02:13:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 -- start start 2024-09-21T02:13:03.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a581fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.759+0000 7f8a5fe5e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a581fbe70 0x7f8a580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a580b5440 0x7f8a5815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58169370 con 0x7f8a44003a40 2024-09-21T02:13:03.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581691f0 con 0x7f8a580b5440 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8a581694f0 con 0x7f8a581fbe70 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a581fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a581fd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35928/0 (socket says 172.21.15.96:35928) 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 -- 172.21.15.96:0/3411246825 learned_addr learned my addr 172.21.15.96:0/3411246825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 -- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a581fbe70 msgr2=0x7f8a580b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a581fbe70 0x7f8a580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 -- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a580b5440 msgr2=0x7f8a5815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a580b5440 0x7f8a5815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 -- 172.21.15.96:0/3411246825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581fd640 con 0x7f8a44003a40 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5dbfa640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a581fd100 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a4801d080 tx=0x7f8a480055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480b1040 con 0x7f8a44003a40 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a48004020 con 0x7f8a44003a40 2024-09-21T02:13:03.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a580b5850 con 0x7f8a44003a40 2024-09-21T02:13:03.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48004450 con 0x7f8a44003a40 2024-09-21T02:13:03.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581f97c0 con 0x7f8a44003a40 2024-09-21T02:13:03.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a580c4ed0 con 0x7f8a44003a40 2024-09-21T02:13:03.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a480be060 con 0x7f8a44003a40 2024-09-21T02:13:03.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a4e7fc640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8a3c073230 0x7f8a3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.763+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f8a4812e640 con 0x7f8a44003a40 2024-09-21T02:13:03.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.767+0000 7f8a5d3f9640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8a3c073230 0x7f8a3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.767+0000 7f8a5d3f9640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8a3c073230 0x7f8a3c075740 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000fc0 tx=0x7f8a54001590 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.767+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8a580b5850 con 0x7f8a44003a40 2024-09-21T02:13:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.939+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 --> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a58073310 con 0x7f8a3c073230 2024-09-21T02:13:03.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.939+0000 7f8a4e7fc640 1 -- 172.21.15.96:0/3411246825 <== mgr.4139 v2:172.21.15.122:6800/3701893359 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13345 (secure 0 0 0) 0x7f8a58073310 con 0x7f8a3c073230 2024-09-21T02:13:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:13:03.945 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-21T02:13:03.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8a3c073230 msgr2=0x7f8a3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8a3c073230 0x7f8a3c075740 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000fc0 tx=0x7f8a54001590 comp rx=0 tx=0).stop 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 msgr2=0x7f8a581fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a581fd100 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a4801d080 tx=0x7f8a480055b0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 shutdown_connections 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f8a3c073230 0x7f8a3c075740 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a580b5440 0x7f8a5815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a581fbe70 0x7f8a580b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 --2- 172.21.15.96:0/3411246825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a44003a40 0x7f8a581fd100 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 >> 172.21.15.96:0/3411246825 conn(0x7f8a5806eea0 msgr2=0x7f8a58159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 shutdown_connections 2024-09-21T02:13:03.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:03.943+0000 7f8a5fe5e640 1 -- 172.21.15.96:0/3411246825 wait complete. 2024-09-21T02:13:03.968 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-21T02:13:02.586390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229844,"kb_used_data":5288,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744876,"statfs":{"total":773094113280,"available":772858753024,"internally_reserved":0,"allocated":5414912,"data_stored":3016408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003520"},"pg_stats":[{"pgid":"1.0","version":"45'74","reported_seq":168,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-21T02:12:43.867671+0000","last_change":"2024-09-21T02:12:43.867671+0000","last_active":"2024-09-21T02:12:43.867671+0000","last_peered":"2024-09-21T02:12:43.867671+0000","last_clean":"2024-09-21T02:12:43.867671+0000","last_became_active":"2024-09-21T02:09:02.622316+0000","last_became_peered":"2024-09-21T02:09:02.622316+0000","last_unstale":"2024-09-21T02:12:43.867671+0000","last_undegraded":"2024-09-21T02:12:43.867671+0000","last_fullsized":"2024-09-21T02:12:43.867671+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'74","last_scrub_stamp":"2024-09-21T02:12:43.867478+0000","last_deep_scrub":"45'74","last_deep_scrub_stamp":"2024-09-21T02:12:43.867478+0000","last_clean_scrub_stamp":"2024-09-21T02:12:43.867478+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:56.763120+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":819827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31708,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":528384,"data_stored":229459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":819827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:03.969 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T02:13:04.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 -- 172.21.15.96:0/3068565357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e0155040 msgr2=0x7f21e00ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 --2- 172.21.15.96:0/3068565357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e0155040 0x7f21e00ca880 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f21e00cc6f0 tx=0x7f21d001c770 comp rx=0 tx=0).stop 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 -- 172.21.15.96:0/3068565357 shutdown_connections 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 --2- 172.21.15.96:0/3068565357 >> v2:172.21.15.122:3300/0 conn(0x7f21e0073800 0x7f21e0073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 --2- 172.21.15.96:0/3068565357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e0155040 0x7f21e00ca880 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 -- 172.21.15.96:0/3068565357 >> 172.21.15.96:0/3068565357 conn(0x7f21e006eea0 msgr2=0x7f21e006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 -- 172.21.15.96:0/3068565357 shutdown_connections 2024-09-21T02:13:04.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 -- 172.21.15.96:0/3068565357 wait complete. 2024-09-21T02:13:04.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 Processor -- start 2024-09-21T02:13:04.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.123+0000 7f21e65c9640 1 -- start start 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 0x7f21e00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21e0073800 0x7f21e014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21e0155040 0x7f21e01f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01661a0 con 0x7f21e006dea0 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0166020 con 0x7f21e0155040 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f21e0166320 con 0x7f21e0073800 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 0x7f21e00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 0x7f21e00cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35938/0 (socket says 172.21.15.96:35938) 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 -- 172.21.15.96:0/726292167 learned_addr learned my addr 172.21.15.96:0/726292167 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e4b66640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21e0155040 0x7f21e01f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21d77fe640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21e0073800 0x7f21e014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 -- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21e0073800 msgr2=0x7f21e014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21e0073800 0x7f21e014cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 -- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21e0155040 msgr2=0x7f21e01f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21e0155040 0x7f21e01f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 -- 172.21.15.96:0/726292167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e01fa430 con 0x7f21e006dea0 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e4b66640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21e0155040 0x7f21e01f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dffff640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 0x7f21e00cf750 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21d00b0040 tx=0x7f21d00a5570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:04.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00bc070 con 0x7f21e006dea0 2024-09-21T02:13:04.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d00043b0 con 0x7f21e006dea0 2024-09-21T02:13:04.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e014d4e0 con 0x7f21e006dea0 2024-09-21T02:13:04.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00b2040 con 0x7f21e006dea0 2024-09-21T02:13:04.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e015e720 con 0x7f21e006dea0 2024-09-21T02:13:04.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.127+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e0073c10 con 0x7f21e006dea0 2024-09-21T02:13:04.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.131+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21d00bf070 con 0x7f21e006dea0 2024-09-21T02:13:04.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.131+0000 7f21dd7fa640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f21c4073200 0x7f21c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.131+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f21d012da00 con 0x7f21e006dea0 2024-09-21T02:13:04.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.131+0000 7f21d77fe640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f21c4073200 0x7f21c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.131+0000 7f21d77fe640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f21c4073200 0x7f21c4075710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f21e00c84d0 tx=0x7f21c8009040 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:04.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.135+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f21d00163f0 con 0x7f21e006dea0 2024-09-21T02:13:04.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.307+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f21e006fa20 con 0x7f21e006dea0 2024-09-21T02:13:04.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.311+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v34)=0 v34) ==== 123+0+0 (secure 0 0 0) 0x7f21d00f8d90 con 0x7f21e006dea0 2024-09-21T02:13:04.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.311+0000 7f21dd7fa640 1 -- 172.21.15.96:0/726292167 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f21d00f81a0 con 0x7f21e006dea0 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f21c4073200 msgr2=0x7f21c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f21c4073200 0x7f21c4075710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f21e00c84d0 tx=0x7f21c8009040 comp rx=0 tx=0).stop 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 msgr2=0x7f21e00cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 0x7f21e00cf750 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21d00b0040 tx=0x7f21d00a5570 comp rx=0 tx=0).stop 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 shutdown_connections 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f21c4073200 0x7f21c4075710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21e0155040 0x7f21e01f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21e0073800 0x7f21e014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 --2- 172.21.15.96:0/726292167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e006dea0 0x7f21e00cf750 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 >> 172.21.15.96:0/726292167 conn(0x7f21e006eea0 msgr2=0x7f21e0151c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 shutdown_connections 2024-09-21T02:13:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.315+0000 7f21e65c9640 1 -- 172.21.15.96:0/726292167 wait complete. 2024-09-21T02:13:04.339 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:13:04.340 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f0979764640 1 -- 172.21.15.96:0/762378622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f095c003a10 msgr2=0x7f095c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f0979764640 1 --2- 172.21.15.96:0/762378622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f095c003a10 0x7f095c003e80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0964009b60 tx=0x7f096401d290 comp rx=0 tx=0).stop 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f09737fe640 1 -- 172.21.15.96:0/762378622 reap_dead start 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f0979764640 1 -- 172.21.15.96:0/762378622 shutdown_connections 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f0979764640 1 -- 172.21.15.96:0/762378622 >> 172.21.15.96:0/762378622 conn(0x7f097406eea0 msgr2=0x7f097406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f0979764640 1 -- 172.21.15.96:0/762378622 shutdown_connections 2024-09-21T02:13:04.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.483+0000 7f0979764640 1 -- 172.21.15.96:0/762378622 wait complete. 2024-09-21T02:13:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 Processor -- start 2024-09-21T02:13:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 -- start start 2024-09-21T02:13:04.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c003a10 0x7f09740c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f09741fc5f0 0x7f0974154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 0x7f09741f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f097416a120 con 0x7f09741556f0 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f097416a2a0 con 0x7f095c003a10 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0979764640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f097416a420 con 0x7f09741fc5f0 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f09737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 0x7f09741f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f09737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 0x7f09741f97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35952/0 (socket says 172.21.15.96:35952) 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f09737fe640 1 -- 172.21.15.96:0/3768537942 learned_addr learned my addr 172.21.15.96:0/3768537942 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f0972ffd640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c003a10 0x7f09740c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.487+0000 7f09727fc640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f09741fc5f0 0x7f0974154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09737fe640 1 -- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f09741fc5f0 msgr2=0x7f0974154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09737fe640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f09741fc5f0 0x7f0974154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09737fe640 1 -- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c003a10 msgr2=0x7f09740c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09737fe640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c003a10 0x7f09740c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09737fe640 1 -- 172.21.15.96:0/3768537942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09741f9000 con 0x7f09741556f0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09727fc640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f09741fc5f0 0x7f0974154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f09737fe640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 0x7f09741f97a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f09740cbb40 tx=0x7f096c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00b7c0 con 0x7f09741556f0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f096c018070 con 0x7f09741556f0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09741fcdd0 con 0x7f09741556f0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00bc40 con 0x7f09741556f0 2024-09-21T02:13:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09741fd390 con 0x7f09741556f0 2024-09-21T02:13:04.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09740cb090 con 0x7f09741556f0 2024-09-21T02:13:04.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f096c01d460 con 0x7f09741556f0 2024-09-21T02:13:04.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f095b7fe640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0960073200 0x7f0960075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f096c093e10 con 0x7f09741556f0 2024-09-21T02:13:04.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.491+0000 7f0972ffd640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0960073200 0x7f0960075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.495+0000 7f0972ffd640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0960073200 0x7f0960075710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09640ae000 tx=0x7f095c005840 comp rx=0 tx=0).ready entity=mgr.4139 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:04.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.495+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f09741f9000 con 0x7f09741556f0 2024-09-21T02:13:04.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09741fcb60 con 0x7f09741556f0 2024-09-21T02:13:04.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f095b7fe640 1 -- 172.21.15.96:0/3768537942 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3914 (secure 0 0 0) 0x7f097406e7f0 con 0x7f09741556f0 2024-09-21T02:13:04.712 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-21T02:13:04.712 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":20,"btime":"2024-09-21T02:12:32:644842+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1509269435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1509269435},{"type":"v1","addr":"172.21.15.122:6835","nonce":1509269435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/815223377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":815223377},{"type":"v1","addr":"172.21.15.122:6837","nonce":815223377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2920253622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2920253622},{"type":"v1","addr":"172.21.15.96:6833","nonce":2920253622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5286,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3723483899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3723483899},{"type":"v1","addr":"172.21.15.96:6835","nonce":3723483899}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-21T02:13:04.713 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 20 2024-09-21T02:13:04.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0960073200 msgr2=0x7f0960075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0960073200 0x7f0960075710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09640ae000 tx=0x7f095c005840 comp rx=0 tx=0).stop 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 msgr2=0x7f09741f97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 0x7f09741f97a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f09740cbb40 tx=0x7f096c00dcd0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 shutdown_connections 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:6800/3701893359,v1:172.21.15.122:6801/3701893359] conn(0x7f0960073200 0x7f0960075710 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09741556f0 0x7f09741f97a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f09741fc5f0 0x7f0974154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 --2- 172.21.15.96:0/3768537942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c003a10 0x7f09740c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.707+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 >> 172.21.15.96:0/3768537942 conn(0x7f097406eea0 msgr2=0x7f097406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.711+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 shutdown_connections 2024-09-21T02:13:04.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-21T02:13:04.711+0000 7f0979764640 1 -- 172.21.15.96:0/3768537942 wait complete. 2024-09-21T02:13:04.735 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T02:13:04.735 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T02:13:04.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:13:04.735 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:04.748 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:13:04.748 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:13:04.749 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:04.808 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:13:04.808 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:13:04.808 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:04.832 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:13:04.832 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:13:04.833 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:04.894 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:13:04.894 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T02:13:04.895 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T02:13:04.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:05.634 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:05.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:07.076 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-21T02:13:10.996 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T02:13:10.997 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T02:13:10.997 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:11.079 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T02:13:11.079 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T02:13:11.079 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:13.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-21T02:13:13.084 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-21T02:13:17.181 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T02:13:17.181 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T02:13:17.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-21T02:13:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-21T02:13:23.283 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T02:13:23.283 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T02:13:23.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:23.341 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T02:13:23.341 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T02:13:23.341 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:23.429 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T02:13:23.429 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T02:13:23.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:24.596 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T02:13:24.596 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-21T02:13:29.530 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T02:13:29.531 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T02:13:29.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:29.612 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T02:13:29.613 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T02:13:29.613 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T02:13:29.613 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:29.647 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T02:13:29.648 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T02:13:29.648 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:29.678 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T02:13:29.678 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T02:13:29.678 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T02:13:29.678 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:29.712 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T02:13:29.713 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T02:13:29.713 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:29.741 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T02:13:29.741 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T02:13:29.741 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:29.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-21T02:13:29.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-21T02:13:29.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-21T02:13:29.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-21T02:13:29.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T02:13:29.798 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T02:13:29.798 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T02:13:29.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T02:13:29.818 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:13:29.818 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T02:13:29.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:13:29.986 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T02:13:30.168 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:13:30.168 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T02:13:30.342 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:13:30.343 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T02:13:30.512 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:13:30.512 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T02:13:30.990 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:13:30.990 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T02:13:31.201 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:13:31.201 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T02:13:31.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:13:31.383 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T02:13:31.533 INFO:tasks.ceph:Archiving mon data... 2024-09-21T02:13:31.568 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/data/mon.a.tgz 2024-09-21T02:13:31.568 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-21T02:13:31.574 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.FqKO7xkuJo 2024-09-21T02:13:31.574 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FqKO7xkuJo 2024-09-21T02:13:31.700 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.FqKO7xkuJo 2024-09-21T02:13:31.767 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.FqKO7xkuJo is 339KB 2024-09-21T02:13:31.829 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.FqKO7xkuJo 2024-09-21T02:13:31.834 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/data/mon.b.tgz 2024-09-21T02:13:31.834 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-21T02:13:31.840 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.q6UIVPjhIv 2024-09-21T02:13:31.840 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.q6UIVPjhIv 2024-09-21T02:13:31.991 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.q6UIVPjhIv 2024-09-21T02:13:32.059 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.q6UIVPjhIv is 452KB 2024-09-21T02:13:32.120 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.q6UIVPjhIv 2024-09-21T02:13:32.125 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/data/mon.c.tgz 2024-09-21T02:13:32.125 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-21T02:13:32.170 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.XlaHtnoENB 2024-09-21T02:13:32.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.XlaHtnoENB 2024-09-21T02:13:32.323 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.XlaHtnoENB 2024-09-21T02:13:32.386 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.XlaHtnoENB is 452KB 2024-09-21T02:13:32.448 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.XlaHtnoENB 2024-09-21T02:13:32.453 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T02:13:32.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T02:13:32.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T02:13:32.516 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T02:13:32.562 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T02:13:32.562 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T02:13:32.564 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/remote/smithi096/crash 2024-09-21T02:13:32.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T02:13:32.579 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/remote/smithi122/crash 2024-09-21T02:13:32.580 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T02:13:32.618 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T02:13:32.618 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:32.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:32.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:32.710 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:32.750 INFO:tasks.ceph:Compressing logs... 2024-09-21T02:13:32.750 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:13:32.756 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:13:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37719.log 2024-09-21T02:13:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36103.log 2024-09-21T02:13:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28892.log 2024-09-21T02:13:32.766 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37719.log.gz 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77424.log 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36103.log.gz 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.28892.log.gz 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77450.log 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77424.log.gz 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38477.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T02:13:32.768 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gz 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.77450.log.gz 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35643.log 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77528.log 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77476.log 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35643.log.gz 2024-09-21T02:13:32.770 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5/var/log/ceph/ceph-client.admin.77528.log: --verbose -- /var/log/ceph/ceph-client.admin.78193.log 2024-09-21T02:13:32.772 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77476.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77528.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78322.log 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77476.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35193.log 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78322.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78193.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28914.log 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78322.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35749.log 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35193.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36302.log 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28914.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35749.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38609.log 2024-09-21T02:13:32.773 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35749.log.gz 2024-09-21T02:13:32.774 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45955.log 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36302.log: /var/log/ceph/ceph-client.0.38609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77933.log 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36302.log.gz 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.45955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45955.log.gz 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77933.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77933.log.gz 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78298.log 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77855.log 2024-09-21T02:13:32.779 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38176.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77855.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.78298.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77855.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29116.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38176.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/ceph-client.admin.36581.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29518.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29116.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36685.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29116.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.38609.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37790.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29518.log.gz 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78011.log 2024-09-21T02:13:32.780 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37790.log.gz 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78011.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.36685.log.gz 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78011.log.gz 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45981.log 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-09-21T02:13:32.781 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.45981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28996.log 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37667.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45981.log.gz 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77959.log 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28996.log.gz 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37693.log 2024-09-21T02:13:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/ceph-client.admin.77959.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38503.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37693.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.77959.log.gz --verbose 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr: -- /var/log/ceph/ceph-client.admin.34968.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37693.log.gz 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34789.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35219.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38503.log.gz 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34968.log.gz 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29092.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37454.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34789.log.gz 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35219.log.gz 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29092.log.gz 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35591.log 2024-09-21T02:13:32.793 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36321.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37454.log.gz 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35449.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35591.log.gz 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/ceph-client.admin.36321.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35449.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36321.log.gz 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35141.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35671.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77907.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35671.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35141.log.gz 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35671.log.gz 2024-09-21T02:13:32.794 INFO:teuthology.orchestra.run.smithi096.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.35449.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35321.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77907.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29032.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38044.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35321.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29032.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78219.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34940.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38044.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77705.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78219.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77398.log 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34940.log.gz 2024-09-21T02:13:32.795 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77829.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77705.log.gz 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78089.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77398.log.gz 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77829.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77829.log.gz 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77627.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78089.log.gz 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35907.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37563.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77627.log.gz 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37537.log 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37563.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35907.log.gz 2024-09-21T02:13:32.796 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36746.log 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37563.log.gz 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38425.log 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37537.log.gz 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78115.log 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36746.log.gz 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78271.log 2024-09-21T02:13:32.797 INFO:teuthology.orchestra.run.smithi096.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38425.log.gz 2024-09-21T02:13:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78141.log 2024-09-21T02:13:32.820 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78115.log.gz 2024-09-21T02:13:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29128.log 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr: 86.8%/var/log/ceph/ceph-client.admin.78141.log: -- replaced with /var/log/ceph/ceph-client.admin.78271.log.gz 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78141.log.gz 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29128.log.gz 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38018.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24025.log 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.tmp-client.admin.24025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35535.log 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24025.log.gz 2024-09-21T02:13:32.821 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5/var/log/ceph/ceph-client.admin.36320.log: --verbose -- /var/log/ceph/ceph-client.admin.38265.log 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35535.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35535.log.gz 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37991.log 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38265.log.gz 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37589.log 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-21T02:13:32.822 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37991.log.gz 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37589.log.gz 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35115.log 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6%/var/log/ceph/ceph-client.admin.28964.log: -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37874.log 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35853.log 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35115.log.gz 2024-09-21T02:13:32.823 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77803.log 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35853.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37874.log.gz 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36659.log 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35853.log.gz 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28984.log 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77803.log.gz 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29464.log 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36659.log.gz/var/log/ceph/ceph-client.admin.28984.log: 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- 86.7% /var/log/ceph/ceph-client.admin.37342.log -- replaced with /var/log/ceph/ceph-client.admin.28984.log.gz 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T02:13:32.824 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29464.log: /var/log/ceph/ceph-client.admin.37342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35167.log 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29464.log.gz 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28852.log 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35167.log.gz 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28852.log.gz 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36087.log 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-09-21T02:13:32.825 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38451.log 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37092.log 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36087.log.gz 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36503.log 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37092.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38451.log.gz 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37092.log.gz 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36318.log 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36503.log.gz 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29611.log 2024-09-21T02:13:32.826 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34822.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz -5 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.0.36871.log 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36318.log: /var/log/ceph/ceph-client.admin.29611.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36318.log.gz 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29611.log.gz 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.0.36871.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36097.log 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28952.log 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36097.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.36871.log.gz 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36097.log.gz 2024-09-21T02:13:32.827 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37127.log 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37205.log 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28952.log: gzip/var/log/ceph/ceph-client.admin.37127.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35089.log 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28952.log.gz 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37205.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37127.log.gz 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29068.log 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37205.log.gz 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35089.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35089.log.gz 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35905.log 2024-09-21T02:13:32.828 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78245.log 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29068.log.gz 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37231.log 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35031.log 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35905.log.gz 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78245.log: /var/log/ceph/ceph-client.admin.37231.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78245.log.gz 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37231.log.gz 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37965.log 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36098.log 2024-09-21T02:13:32.829 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35031.log: /var/log/ceph/ceph-client.admin.37965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38291.log 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35031.log.gz 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36098.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37965.log.gz 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36098.log.gz 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38321.log 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28898.log 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38291.log: /var/log/ceph/ceph-client.admin.38321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35245.log 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38291.log.gz 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38321.log.gz 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28898.log.gz 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37842.log 2024-09-21T02:13:32.830 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37257.log 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35245.log.gz/var/log/ceph/ceph-client.admin.37842.log: 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37842.log.gz 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37257.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37257.log.gz 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77580.log 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29044.log 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77881.log 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77580.log: /var/log/ceph/ceph-client.admin.29044.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77580.log.gz 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29044.log.gz 2024-09-21T02:13:32.831 INFO:teuthology.orchestra.run.smithi096.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T02:13:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35903.log 2024-09-21T02:13:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37309.log 2024-09-21T02:13:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34325.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.1.34325.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 90.8% --verbose -- replaced with /var/log/ceph/ceph-client.1.34325.log.gz 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/ceph-client.admin.29006.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29109.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29085.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29006.log.gz 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34444.log 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr: 86.5%/var/log/ceph/ceph-client.admin.29085.log: -- replaced with /var/log/ceph/ceph-client.admin.29109.log.gz 2024-09-21T02:13:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29085.log.gz 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.3.34444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29061.log 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29157.log 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29061.log: 90.7% -- replaced with /var/log/ceph/ceph-client.3.34444.log.gz 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29030.log 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29061.log.gz 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29133.log 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29157.log.gz 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29030.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T02:13:32.847 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29030.log.gz 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29133.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29133.log.gz 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5%/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T02:13:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T02:13:32.849 INFO:teuthology.orchestra.run.smithi122.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T02:13:32.849 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.b.log: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T02:13:32.849 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28982.log 2024-09-21T02:13:32.849 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.28982.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28982.log.gz 2024-09-21T02:13:32.863 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77881.log: /var/log/ceph/ceph-client.admin.35903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28940.log 2024-09-21T02:13:32.863 INFO:teuthology.orchestra.run.smithi096.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.77881.log.gz 2024-09-21T02:13:32.863 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5%/var/log/ceph/ceph-client.admin.37309.log: -- replaced with /var/log/ceph/ceph-client.admin.35903.log.gz 2024-09-21T02:13:32.863 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37309.log.gz 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78167.log 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36633.log 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28940.log.gz 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77653.log 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:gzip 87.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.35617.log 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78167.log.gz/var/log/ceph/ceph-client.admin.36633.log: 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77653.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36633.log.gz 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77653.log.gz 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37179.log 2024-09-21T02:13:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35617.log.gz/var/log/ceph/ceph-client.admin.37179.log: 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77777.log 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37179.log.gz 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35775.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gz 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36921.log 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77777.log: /var/log/ceph/ceph-client.admin.36921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38553.log 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77777.log.gz 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36921.log.gz 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38204.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2024-09-21T02:13:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37615.log 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29440.log 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38553.log.gz/var/log/ceph/ceph-client.admin.37615.log: 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37615.log.gz 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29440.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29440.log.gz 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77679.log 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34731.log 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29080.log 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77679.log: /var/log/ceph/ceph-client.admin.34731.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77679.log.gz 2024-09-21T02:13:32.866 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34731.log.gz 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35899.log 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77749.log 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29080.log.gz 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35899.log: /var/log/ceph/ceph-client.admin.77749.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35899.log.gz 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.77749.log.gz 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35906.log 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36114.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr: 87.5%/var/log/ceph/ceph-client.admin.35906.log: -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T02:13:32.867 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35906.log.gz 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29104.log 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr: 88.9%/var/log/ceph/ceph-client.admin.29104.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29104.log.gz 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78348.log 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38239.log 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78348.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78348.log.gz 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77554.log 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38239.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38239.log.gz 2024-09-21T02:13:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.37016.log 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77554.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.77554.log.gz 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35827.log 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.2.37016.log: 91.5% -- replaced with /var/log/ceph/ceph-client.2.37016.log.gz 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35827.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35827.log.gz 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36295.log 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37481.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36295.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36295.log.gz 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36104.log 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34888.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-09-21T02:13:32.869 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36317.log 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36104.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36104.log.gz 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34914.log 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36317.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36317.log.gz 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37939.log 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34914.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34914.log.gz 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38070.log 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37939.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37939.log.gz 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38070.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38070.log.gz 2024-09-21T02:13:32.870 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37816.log 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36529.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29056.log 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37816.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37816.log.gz 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38148.log 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29056.log.gz 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38122.log 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38148.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38148.log.gz 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77985.log 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38122.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38122.log.gz 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38373.log 2024-09-21T02:13:32.871 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.77985.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77985.log.gz 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35061.log 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38373.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38373.log.gz 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35061.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35061.log.gz 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78037.log 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37066.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29008.log 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78037.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78037.log.gz 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-09-21T02:13:32.872 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.29008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29008.log.gz 2024-09-21T02:13:32.873 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-09-21T02:13:32.873 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37745.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-09-21T02:13:32.873 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35801.log 2024-09-21T02:13:32.880 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36323.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-09-21T02:13:32.880 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38399.log 2024-09-21T02:13:32.880 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35801.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35801.log.gz 2024-09-21T02:13:32.880 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-21T02:13:32.880 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38399.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38399.log.gz 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35697.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78063.log 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35902.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.78063.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78063.log.gz 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37283.log 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37641.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35893.log 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37283.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37283.log.gz 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36303.log 2024-09-21T02:13:32.881 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35893.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35893.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35508.log 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28849.log 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36303.log: /var/log/ceph/ceph-client.admin.35508.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36303.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35508.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35271.log 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28935.log 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28849.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37153.log 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.28935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28935.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.35271.log.gz 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77502.log 2024-09-21T02:13:32.882 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38347.log 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37153.log: /var/log/ceph/ceph-client.admin.77502.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37153.log.gz 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77502.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38347.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38347.log.gz 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35565.log 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35904.log: /var/log/ceph/ceph-client.admin.35565.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gz 2024-09-21T02:13:32.883 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35565.log.gz 2024-09-21T02:13:32.889 INFO:teuthology.orchestra.run.smithi096.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T02:13:32.896 INFO:teuthology.orchestra.run.smithi122.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T02:13:32.942 INFO:teuthology.orchestra.run.smithi122.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T02:13:33.008 INFO:teuthology.orchestra.run.smithi122.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T02:13:33.126 INFO:teuthology.orchestra.run.smithi096.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T02:13:33.461 INFO:teuthology.orchestra.run.smithi096.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T02:13:33.491 INFO:teuthology.orchestra.run.smithi096.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T02:13:33.520 INFO:teuthology.orchestra.run.smithi122.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T02:13:33.529 INFO:teuthology.orchestra.run.smithi096.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T02:13:33.545 INFO:teuthology.orchestra.run.smithi122.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T02:13:33.583 INFO:teuthology.orchestra.run.smithi122.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T02:13:33.601 INFO:teuthology.orchestra.run.smithi122.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T02:13:33.602 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-21T02:13:33.602 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m0.801s 2024-09-21T02:13:33.602 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m3.208s 2024-09-21T02:13:33.602 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.145s 2024-09-21T02:13:33.604 INFO:teuthology.orchestra.run.smithi096.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T02:13:33.606 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-21T02:13:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m0.848s 2024-09-21T02:13:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m3.374s 2024-09-21T02:13:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.214s 2024-09-21T02:13:33.606 INFO:tasks.ceph:Archiving logs... 2024-09-21T02:13:33.607 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/remote/smithi096/log 2024-09-21T02:13:33.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:13:34.161 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/remote/smithi122/log 2024-09-21T02:13:34.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:13:34.544 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T02:13:34.553 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T02:13:34.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:13:34.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:13:34.632 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T02:13:34.641 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T02:13:34.641 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:13:34.646 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:13:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:13:34.655 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-09-21T02:13:34.655 INFO:teuthology.orchestra.run.smithi096.stdout:*hv01.front.sepi 67.205.162.81 3 u 34 64 377 0.075 +1.049 0.185 2024-09-21T02:13:34.655 INFO:teuthology.orchestra.run.smithi096.stdout:+hv02.front.sepi 63.231.80.2 3 u 28 64 377 0.071 +1.869 0.178 2024-09-21T02:13:34.655 INFO:teuthology.orchestra.run.smithi096.stdout:+hv03.front.sepi 74.6.168.72 3 u 27 64 377 0.088 +0.492 0.292 2024-09-21T02:13:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:13:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:13:34.659 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-21T02:13:34.659 INFO:teuthology.orchestra.run.smithi122.stdout:*hv01.front.sepi 67.205.162.81 3 u 29 64 377 0.118 -0.970 0.554 2024-09-21T02:13:34.659 INFO:teuthology.orchestra.run.smithi122.stdout:+hv02.front.sepi 63.231.80.2 3 u 24 64 377 0.130 -0.200 0.559 2024-09-21T02:13:34.659 INFO:teuthology.orchestra.run.smithi122.stdout:+hv03.front.sepi 74.6.168.72 3 u 29 64 377 0.102 -1.682 0.566 2024-09-21T02:13:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:13:34.660 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T02:13:34.668 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T02:13:34.669 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T02:13:34.677 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T02:13:34.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T02:13:34.692 INFO:teuthology.task.internal:Duration was 945.999045 seconds 2024-09-21T02:13:34.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T02:13:34.701 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T02:13:34.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:13:34.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:13:34.738 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T02:13:34.738 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-21T02:13:34.738 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:13:34.792 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T02:13:34.792 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:13:34.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-21T02:13:34.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-21T02:13:34.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-21T02:13:34.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-21T02:13:34.801 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T02:13:34.802 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:13:34.835 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:13:34.875 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T02:13:34.875 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:13:34.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:13:34.974 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T02:13:34.983 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T02:13:34.983 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:13:34.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:13:35.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T02:13:35.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:13:35.039 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:13:35.048 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-21T02:13:35.060 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-21T02:13:35.073 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:13:35.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:13:35.106 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:13:35.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:13:35.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T02:13:35.128 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T02:13:35.128 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/remote/smithi096 2024-09-21T02:13:35.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:13:35.170 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913407/remote/smithi122 2024-09-21T02:13:35.170 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:13:35.191 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T02:13:35.191 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:13:35.208 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:13:35.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T02:13:35.244 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T02:13:35.245 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T02:13:35.254 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T02:13:35.254 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:13:35.256 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:13:35.259 INFO:teuthology.orchestra.run.smithi096.stdout: 658640 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 21 02:13 /home/ubuntu/cephtest 2024-09-21T02:13:35.259 INFO:teuthology.orchestra.run.smithi096.stdout: 683136 4 d--------- 2 ubuntu ubuntu 4096 Sep 21 02:10 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:35.259 INFO:teuthology.orchestra.run.smithi096.stderr:find: '/home/ubuntu/cephtest/mnt.0': Permission denied 2024-09-21T02:13:35.261 INFO:teuthology.orchestra.run.smithi096.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-09-21T02:13:35.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:13:35.279 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi096 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-21T02:13:35.280 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T02:13:35.296 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T02:13:35.316 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi096 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-21T02:13:35.318 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/acls} duration: 945.9990448951721 failure_reason: 'Command failed on smithi096 with status 1: ''find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest''' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-09-21T02:13:35.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:13:35.323 INFO:teuthology.orchestra.run.smithi122.stdout: 658622 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:13 /home/ubuntu/cephtest 2024-09-21T02:13:35.426 INFO:teuthology.run:FAIL